BERT in bert-for-tf2 is implemented as a Keras layer. You could instantiate it like this: frombertimportBertModelLayerl_bert=BertModelLayer(**BertModelLayer.Params(vocab_size=16000,# embedding paramsuse_token_type=True,use_position_embeddings=True,token_type_vocab_size=2,num_layers=12,# tran...
Here is my code snippets max_seq_len = 128 bert_params = bert.params_from_pretrained_ckpt(model_dir) bert_layer = bert.BertModelLayer.from_params(bert_params, name="bert") input_ids = Input(shape=(max_seq_len,), dtype='int32') masked = M...