Missing tokenizer file
#1
by
iiivyliu
- opened
The tokenizer files for this model seem to be missing, which prevents it from being loaded with the standard transformers.pipeline. Could you please consider uploading them to make the model usable?
there is actually no tokenizer you have to pretokenize the data some other way