これは、なにをしたくて書いたもの？
以前のエントリーで、SentenceTransformersとintfloat/multilingual-e5のモデルを使ってテキスト埋め込みを試してみました。
SentenceTransformersとintfloat/multilingual-e5でテキスト埋め込みを試してみる - CLOVER🍀
intfloat/multilingual-e5のモデルを使うと日本語にも効果的なようなので、便利なのですが、SentenceTransformersを使っていると
Pythonからしか使えません。
となるとOpenAI API経由で使いたいなと思い、今度はLocalAIで試してみることにしました。
これができると、Chat APIはllama.appで動かし、テキスト埋め込みはSentenceTransfomersで動かすといった利用するモデルに応じた
使い分けがLocalAIでできるようになるな、と。
LocalAI × SentenceTransformers
LocalAIはバックエンドにいくつかのモデルをサポートしており、その中にSentenceTransformersが含まれています。
Model compatibility :: LocalAI documentation
Embeddingsのみで使えるバックエンドとなっています。
使い方はEmbeddingsのページに書かれているので、こちらに沿って環境を作って試してみたいと思います。
🧠 Embeddings :: LocalAI documentation
今回は頑張って環境を作りましたが、Dockerイメージ版を使うのがトラブらなくてよいかなと思いましたが…。
環境
今回の環境はこちら。
$ python3 --version Python 3.10.12 $ pip3 --version pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
使用するLocalAIのバージョンはこちら。
$ ./local-ai-avx2-Linux-x86_64 --version LocalAI version v2.4.1 (ce724a7e555f840929bd001dc0148aee69da9a1f)
CPU環境で実行します。
SentenceTransformersをバックエンドにする場合の前提条件を確認する
LocalAIのバックエンドにSentenceTransformersを使う場合、いろいろと注意事項が書かれているので確認してみます。
どうやら他のバックエンドと異なり、Pythonのソースコードを直接実行するようになっているようです。
To use sentence-transformers and models in huggingface you can use the sentencetransformers embedding backend.
Embeddings / Huggingface embeddings
なので、実行にはPytyhonが必要です。
The sentencetransformers backend is an optional backend of LocalAI and uses Python.
依存関係は自分でインストールする必要があります。condaの利用を前提としているようです。
If you are running LocalAI manually you must install the python dependencies (make prepare-extra-conda-environments). This requires conda to be installed.
ローカル実行の場合は、環境変数EXTERNAL_GRPC_BACKENDSに実行するスクリプトを指定する必要があります。
For local execution, you also have to specify the extra backend in the EXTERNAL_GRPC_BACKENDS environment variable.
例はこちらです。
EXTERNAL_GRPC_BACKENDS="sentencetransformers:/path/to/LocalAI/backend/python/sentencetransformers/sentencetransformers.py"
LocalAIを実行するのはローカルなのでは…？と思い、これは最初なにを言っているのかわからなかったのですが、これはDockerイメージ版と
比較した時のことを言っていますね。
Dockerfileでは、このあたりは指定済みです。
https://github.com/mudler/LocalAI/blob/v2.4.1/Dockerfile#L16
実行に指定するスクリプトはまた後で。
SentenceTransformersバックエンドはテキスト埋め込みのみをサポートしており、トークンの埋め込みはサポートしていません。
トークンの埋め込みが必要な場合は、llama.cppまたはbert.cppを使うことになります。
The sentencetransformers backend does support only embeddings of text, and not of tokens. If you need to embed tokens you can use the bert backend or llama.cpp.
テキストの埋め込みはいいとして、トークンの埋め込みとは？と思ったのですが、テキストをトークン化して、さらにそのトークンの
ベクトルを求めることを言うんでしょうか…？
What Are Transformer Models and How Do They Work?
モデルは事前にダウンロードする必要はなく、SentenceTransformersが自動的にHugging Face Hubからダウンロードしてきます。
No models are required to be downloaded before using the sentencetransformers backend. The models will be downloaded automatically the first time the API is used.
というわけで、SentenceTransformersをテキスト埋め込みのバックエンドにする場合はDockerイメージ版を使った方が環境設定としては
簡単になります。PythonおよびSentenceTransformersがインストール済み、環境変数も設定済み、となるので。
用意されている（環境別の）Dockerイメージは、こちらにリストアップされています。
Getting Started / Container images
なのですが、今回はローカルでバイナリを実行する方向でいきたいと思います。
SentenceTransformersをバックエンドとして実行できるように設定する
SentenceTransformersを、LocalAIのバックエンドとして実行できるように設定していきます。
ドキュメントではEXTERNAL_GRPC_BACKENDS環境変数には以下の値を指定するように書かれているのですが、
EXTERNAL_GRPC_BACKENDS="sentencetransformers:/path/to/LocalAI/backend/python/sentencetransformers/sentencetransformers.py"
Dockerfileを見ると指定されている値は以下のようになっています。
sentencetransformers:/build/backend/python/sentencetransformers/run.sh
https://github.com/mudler/LocalAI/blob/v2.4.1/Dockerfile#L16
スクリプトを確認すると、環境設定をしてsentencetransformers/sentencetransformers.pyを実行しているようです。
https://github.com/mudler/LocalAI/blob/v2.4.1/backend/python/sentencetransformers/run.sh
condaは使っていないので、このあたりは仮想環境に読み替えても良さそうな気がしますね。
では、Pythonはインストール済みなので、必要なライブラリーをインストールするところから始めていきます。
こちらを確認すると、依存関係は実行環境に応じてtransformers.ymlまたはtransformers-nvidia.ymlに書かれているようです。
https://github.com/mudler/LocalAI/blob/v2.4.1/backend/python/common-env/transformers/Makefile
transformers.ymlに書かれている依存関係を使うことにします。
仮想環境の作成、有効化。
$ python3 -m venv venv $ . venv/bin/activate
ここからrequirements.txtを作成。
$ curl -s -L https://github.com/mudler/LocalAI/raw/v2.4.1/backend/python/common-env/transformers/transformers.yml | grep '^ - ' | perl -wp -e 's!^ - !!' > requirements.txt
こうなりました。
requirements.txt
accelerate==0.23.0 aiohttp==3.8.5 aiosignal==1.3.1 async-timeout==4.0.3 attrs==23.1.0 bark==0.1.5 boto3==1.28.61 botocore==1.31.61 certifi==2023.7.22 TTS==0.22.0 charset-normalizer==3.3.0 datasets==2.14.5 sentence-transformers==2.2.2 sentencepiece==0.1.99 dill==0.3.7 einops==0.7.0 encodec==0.1.1 filelock==3.12.4 frozenlist==1.4.0 fsspec==2023.6.0 funcy==2.0 grpcio==1.59.0 huggingface-hub==0.16.4 idna==3.4 jinja2==3.1.2 jmespath==1.0.1 markupsafe==2.1.3 mpmath==1.3.0 multidict==6.0.4 multiprocess==0.70.15 networkx numpy==1.26.0 packaging==23.2 pandas peft==0.5.0 git+https://github.com/bigscience-workshop/petals protobuf==4.24.4 psutil==5.9.5 pyarrow==13.0.0 python-dateutil==2.8.2 pytz==2023.3.post1 pyyaml==6.0.1 regex==2023.10.3 requests==2.31.0 rouge==1.0.1 s3transfer==0.7.0 safetensors==0.3.3 scipy==1.11.3 six==1.16.0 sympy==1.12 tokenizers==0.14.0 torch==2.1.0 torchaudio==2.1.0 tqdm==4.66.1 transformers==4.34.0 triton==2.1.0 typing-extensions==4.8.0 tzdata==2023.3 urllib3==1.26.17 xxhash==3.4.1 yarl==1.9.2 soundfile langid wget unidecode pyopenjtalk-prebuilt pypinyin inflect cn2an jieba eng_to_ipa openai-whisper matplotlib gradio==3.41.2 nltk sudachipy sudachidict_core vocos
ライブラリーのインストール。
$ pip3 install -r requirements.txt
すると、依存関係の解決に失敗しました…。
ERROR: Cannot install -r requirements.txt (line 1), -r requirements.txt (line 10), -r requirements.txt (line 6) and numpy==1.26.0 because these package versions have conflicting dependencies.
The conflict is caused by:
The user requested numpy==1.26.0
accelerate 0.23.0 depends on numpy>=1.17
bark 0.1.5 depends on numpy
tts 0.22.0 depends on numpy==1.22.0; python_version <= "3.10"
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
LocalAIのDockerfileで構築される環境はPython 3.9になるみたいなので、ローカルのPythonが新しいみたいですね…。
TTSと合わないみたいなのと、TTSはText-to-Speechで使うもののようなので今回は要らない気がします。なのでTTSのバージョン固定を
外そうと思ったのですが、外してみるとnumpyが他のいろんなところでひっかかったので、numpyのバージョン固定を解除することに
しました…。
なので、こちらを
numpy==1.26.0
こう変更。
numpy
再度インストールして完了。
$ pip3 install -r requirements.txt
こういうことがあると、素直にDockerイメージ版を使った方がいい気がしますね…。
ディスクサイズはこの程度に。
$ du -sh venv 6.8G venv
実行にはsentencetransformers.pyが必要になるので、ソースコードをダウンロードして展開します。
$ curl -LO https://github.com/mudler/LocalAI/archive/refs/tags/v2.4.1.tar.gz $ tar xf v2.4.1.tar.gz
必要なファイルはこちらに含まれています。
$ tree LocalAI-2.4.1/backend/python/sentencetransformers LocalAI-2.4.1/backend/python/sentencetransformers ├── Makefile ├── README.md ├── backend_pb2.py ├── backend_pb2_grpc.py ├── run.sh ├── sentencetransformers.py ├── test.sh └── test_sentencetransformers.py 0 directories, 8 files
環境変数EXTERNAL_GRPC_BACKENDSを以下のように設定。
$ export EXTERNAL_GRPC_BACKENDS=sentencetransformers:LocalAI-2.4.1/backend/python/sentencetransformers/sentencetransformers.py
設定ファイルはこういう感じで用意しました。
local-ai-config.yaml
- name: intfloat-multilingual-e5-base backend: sentencetransformers embeddings: true parameters: model: intfloat/multilingual-e5-base
起動。ちなみに、modelsディレクトリの中身は空です。
$ ./local-ai-avx2-Linux-x86_64 --config-file local-ai-config.yaml --models-path models --threads 4
動作確認。
$ curl -XPOST -s -H 'Content-Type: application/json' localhost:8080/v1/embeddings -d '{"input": "query: Your text string goes here", "model": "intfloat-multilingual-e5-base"}'
初回は特に時間がかかりますが、結果が返ってきました。
※モデルがダウンロードされていない場合は、SentenceTransformersが（というかTransformersが）ダウンロードしてきます
{"created":1704688219,"object":"list","id":"4d8b92a2-ff8a-4694-bcae-6aed293da294","model":"intfloat-multilingual-e5-base","data":[{"embedding":[0.025812576,0.038589943,-0.0058456543,0.033882763,0.016148956,-0.05467815,-0.029496888,-0.046939034,0.010684317,0.04915923,0.003973063,0.0036892025,0.13803554,0.018343633,-0.019557545,-0.024634045,-0.0041345484,-0.03301773,0.024800006,-0.0067035705,0.059539225,-0.026738506,0.00041379395,-0.017696682,0.010480808,-0.035041608,0.022870634,0.04802763,-0.011051078,0.06849486,0.03837306,-0.0074592875,0.015809722,0.037073493,0.033075634,0.053927604,-0.017501533,-0.018305713,0.036818072,0.012861759,-0.009304923,0.05107057,0.0337262,-0.041696142,0.04898302,-0.036074337,0.011346245,0.029628558,-0.027535904,-0.035001334,0.04152875,0.008216662,0.0015923717,0.02791197,-0.063504495,-0.037630904,0.015885646,0.022621805,0.00611842,0.047560386,-0.009098981,0.050055545,-0.03794484,0.066165686,0.036689978,-0.034246214,0.0048117437,-0.029768027,-0.059477195,-0.026592188,0.0011505693,-0.020896627,0.015097429,0.006395011,-0.05467136,-0.037914876,-0.04177243,0.007988624,-0.004592513,-0.016932573,0.059998125,0.037247628,0.055698864,0.03615363,-0.025697986,0.01094647,-0.0074477983,0.033990487,0.012622544,0.059534267,0.011737165,-0.002301323,-0.050740782,0.049013883,-0.014756835,0.019945934,0.027406858,0.010481469,0.05955573,-0.03000983,-0.04523292,-0.07383306,-0.021207439,-0.08501222,-0.06467389,-0.004954412,0.011838736,-0.049207237,0.019940969,-0.008521365,-0.032986168,0.019763801,0.0579679,-0.04019426,0.014643791,-0.015789373,0.041887254,-0.017992424,0.0066412645,-0.041882858,0.037376624,-0.006374157,-0.017009482,-0.00026709939,0.038719032,-0.027745657,0.0099800145,-0.0277918,0.06540708,-0.055828374,-0.025301956,-0.010120397,-0.034420602,0.03468939,-0.03251213,0.032352548,0.04048519,0.0047621573,0.0023064301,-0.003055169,0.025334954,-0.051892247,-0.0050340397,0.04384306,0.052508287,-0.05408677,0.0052595967,0.00705437,-0.0061318036,0.045001667,0.04913167,-0.040982258,-0.011798139,-0.025425533,0.050905466,-0.0037612144,-0.030549686,0.0042121583,0.0041592713,0.03820377,0.03416491,-0.0073974794,0.008080944,-0.016424771,0.04030189,-0.008360291,-0.015012127,-0.0071074395,-0.019551164,-0.035554364,-0.0117765535,-0.028658729,-0.038770456,0.003006043,0.000189157,-0.020379623,-0.016019262,-0.049648546,-0.0033587078,-0.02343635,-0.051786467,-0.021952808,-0.040376868,0.035526134,-0.0029035227,-0.0072268415,-0.038164854,-0.012750922,0.034338728,-0.007576396,-0.029850937,0.062980466,0.03132553,0.0045848344,0.010636684,0.023381533,0.008755578,0.03252862,-0.008851995,-0.045052204,0.014203277,-0.016178476,0.029963765,-0.0020236494,-0.0372839,-0.0619141,0.0013257547,0.049174663,0.046473034,0.04070244,-0.019113045,0.0520985,-0.0171347,0.0040515126,-0.027849214,-0.022366412,0.046544798,-0.010206897,-0.017768819,0.0239627,0.049277406,-0.064892724,0.058383275,0.027522776,0.0062270937,-0.00506276,0.010556536,-0.0041673253,0.014889858,0.044701546,0.030812133,0.038925957,-0.035345413,0.047703374,0.009889973,-0.05540426,0.04359623,0.006142385,-0.0032847684,-0.11282283,-0.016040854,0.029058019,0.008643252,-0.024769928,0.0269186,-0.007970509,-0.058334135,0.021654548,-0.075075306,0.027301693,0.011136328,0.0042055217,-0.0031781376,-0.0028842392,0.016179161,-0.010086236,-0.042272862,-0.0033532584,0.009217631,0.05907421,-0.00079532614,-0.05001243,-0.009262884,-0.04050923,0.0154971015,0.04831966,0.08336695,-0.018006843,-0.026492765,-0.029077884,-0.05257872,-0.0037508318,-0.04109274,0.047035597,-0.0049601668,-0.02544683,-0.057264656,0.017644584,-0.06252525,-0.011540723,-0.03442196,0.06518383,-0.07055196,-0.002378635,-0.040327627,0.004131796,-0.09190948,-0.03651089,-0.055274967,-0.0019633397,0.0479862,0.0748624,-0.0072427145,0.0012795324,0.055830996,0.026971519,0.035256058,0.0093947025,0.0030232936,0.042855233,0.011406333,-0.03811503,0.028959312,-0.055968992,-0.020733658,0.013873693,0.105665706,0.023706993,-0.032526303,0.03222356,-0.023147048,-0.011241927,-0.042866528,0.036656577,0.044392515,-0.07957676,0.0053743576,0.06864603,0.0020481206,0.005177826,-0.004196364,0.033119995,-0.01415184,-0.07774955,-0.0020402558,-0.036048084,0.03921993,0.009260698,0.039204217,0.016379144,-0.024666563,0.0056990935,-0.01833019,0.0290113,-0.049629148,0.016360274,-0.020565823,-0.03759553,0.058699865,0.0077293566,-0.012169825,0.043718692,-0.014141721,-0.008107657,0.062090274,-0.04696994,-0.022616815,0.011270156,-0.042588394,0.032622688,0.06680576,-0.03507376,0.03870236,-0.03575712,-0.0029839983,-0.011635301,0.020921348,0.011159883,-0.01618732,-0.006669244,-0.016842552,0.029329982,0.009159299,-0.036865357,0.016476307,-0.002963858,-0.008847507,-0.025787598,0.028004143,-0.027016459,0.069377735,0.01552319,-0.01509847,-0.027861394,0.014009297,-0.065384634,0.0396776,-0.06790957,-0.026972873,0.047350865,0.036258355,0.022930022,0.008213924,0.012219475,-0.031674232,0.02103102,0.05940826,-0.053128894,0.05724853,-0.017011441,0.01638965,-0.04560822,-0.025320292,0.022010865,0.017441383,-0.011279351,0.03394102,-0.008508563,0.0068405718,-0.016698977,0.033656545,0.012902659,0.017781828,0.031645823,0.023276422,0.032513313,-0.027419513,0.04126998,0.0042673554,-0.010056589,0.073989876,-0.046647977,0.04765223,0.011169265,0.007751288,-0.027231328,-0.030436259,0.03179438,0.02917608,0.012183451,-0.051428393,0.032716244,0.01569398,0.04335574,0.011375122,-0.083776064,-0.008535772,-0.015185153,0.027934188,-0.019507434,-0.009674186,-0.031253856,0.02420099,-0.02492535,0.026839254,-0.039520912,0.01717677,0.02528442,-0.03479243,0.019780854,-0.0038799655,-0.016603826,0.023368945,-0.038716797,-0.017830307,-0.021262852,-0.035752703,0.018062131,-0.023486126,-0.01876794,0.039033018,0.0052813557,0.0148402145,-0.018938184,0.04935713,-0.024190038,0.055253107,-0.026372248,0.017894426,-0.08428314,-0.0040355558,0.05053626,-0.035705306,-0.0468893,-0.042400707,-0.057409804,0.00094671064,-0.0626862,0.0125870025,-0.008307095,-0.019675586,-0.02394082,0.0145641435,0.032548215,0.025134543,0.02818987,-0.015568171,0.015791837,0.043027498,-0.028090047,0.006928482,-0.025587203,-0.0516499,-0.007690564,0.02866056,0.018972857,-0.09486287,0.022691863,0.012341029,-0.030351883,-0.0077107735,0.044310734,0.01593274,-0.013648724,-0.0006820999,0.0065152883,-0.030389437,-0.06577994,0.032805234,0.03593632,0.002373279,-0.0386551,-0.023566581,0.0020135874,-0.0017487815,0.0036527663,-0.057164222,0.008088731,0.034525707,-0.0023021393,0.12362234,-0.06660443,-0.020203786,-0.04969978,0.030374946,0.013115277,0.005309359,0.050141905,0.0033380147,-0.00046058904,-0.009688368,0.0068241684,-0.056448396,-0.018654743,0.041125104,-0.000552116,0.015952684,-0.055758495,0.020068236,-0.03266815,0.032614615,-0.05194463,0.019891372,-0.11402178,-0.016773637,0.04179351,-0.01916357,-0.0069640893,0.02445135,-0.021983903,0.014625048,-0.03510115,0.0024515442,-0.018742938,0.025850132,-0.025523096,0.0029397386,-0.0082708895,0.016614804,-0.0014736081,0.015514519,-0.05404993,-0.0009460784,0.07990308,-0.006692739,0.04627013,-0.00031194923,0.034940466,0.01260544,-0.0053506894,-0.008675954,-0.034948464,0.0020521707,0.039550368,-0.010357138,0.0282926,0.0047057993,-0.0023719287,-0.012611004,0.029341426,-0.04558982,0.01971927,0.031482335,-0.05498593,0.043887,0.016306225,-0.064102314,-0.02782977,0.047216263,-0.06814373,-0.03678817,-0.04154542,-0.047680173,0.017880963,0.04635416,0.026907587,0.014369134,-0.021327602,-0.2313437,0.041604497,-0.018115483,0.006459436,-0.0011430788,0.0152692385,0.02323358,-0.012222185,0.01988759,-0.017270166,0.030708138,-0.0641795,0.041880358,-0.023753747,-0.038454454,-0.023393052,-0.012484665,-0.023501981,0.055444296,0.02911461,0.017490871,0.027517803,-0.040599868,-0.022850493,-0.028230816,0.005084518,-0.04299986,0.029891685,0.010063361,0.02065747,-0.009652797,0.03197802,0.016879441,0.04512468,-0.03323426,0.009967964,-0.004604228,0.018658618,-0.07760165,0.011696565,-0.009854091,-0.039735552,-0.07363512,0.03810855,0.0050398717,0.05260615,-0.0066893436,0.0070081223,-0.030869594,-0.01899549,0.029885884,0.021950403,-0.10396562,0.027594885,0.00048609136,0.055941813,0.0032381923,0.047850944,0.046738468,0.0050747637,0.051740695,0.053393424,0.0101794,0.029528242,-0.00079797005,0.0069905384,0.011309334,-0.023557985,-0.049997866,-0.0033189433,0.027624898,-0.024594389,-0.06627349,0.030224372,0.019527672,-0.04752918,0.039272875,0.0019240177,-0.05584772,0.026936432,0.0039381385,-0.01493621,-0.029655792,0.028686637,-0.027922943,-0.0494121,-0.02095978,0.010595278,0.019170769,0.008152632,-0.031931426,-0.01604027,0.00007957141,0.018718626,0.029589687,0.00054057856,-0.022332985,0.028041048,0.017090641,-0.020222919,0.011985957,-0.024905108,-0.04994295,0.026946167,0.053436626,-0.030175483,-0.053468965,0.043297518,-0.01794444,-0.05255278,0.021384582,0.050462224,0.036897358,-0.035610747,0.0024968514,-0.0270824,0.024296334,-0.047144145,-0.025721146,-0.046777766,-0.011098485,-0.03609157,0.022562753,0.010368517,0.0711507,-0.004159833,-0.024059514,0.026424596,0.035312336,0.025277894,-0.038301464,-0.010641833,0.052815855,-0.0529311,-0.037018254,0.03912192,0.003514738,-0.0135293985,0.029693257,0.04014519,0.06739977,-0.09676697,-0.014978698,0.038347982,-0.041015796,0.011303062,0.03574727,-0.027664077,0.03432266,-0.018966634,-0.024944631,-0.08056069,-0.010082288,-0.01236195,-0.01369652,0.020922214,0.017581671,0.0008327719,-0.020025678,0.014481863,-0.014281548,-0.045626223,0.061332993,0.015830688,0.01651178,0.031793296,-0.02642741,-0.0003601319,0.031285796,0.025037326,-0.031499382,0.044825517,0.026612382,0.01510707,0.03606671,-0.022823626,0.06814292,-0.044726305,-0.06495744,0.045035888],"index":0,"object":"embedding"}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
成功したようです。
相変わらず、usageの中身は空なのですが…。
"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}
OpenAI Python APIライブラリーからアクセスしてみる
次に、OpenAI Python APIライブラリーからアクセスしてみましょう。
SentenceTransformersを直接使って書いた以前のエントリーで行ったことを、OpenAI Python APIライブラリーに差し替えて
行ってみたいと思います。
SentenceTransformersとintfloat/multilingual-e5でテキスト埋め込みを試してみる - CLOVER🍀
ライブラリーのインストール。
$ pip3 install openai numpy
依存関係。
$ pip3 list Package Version ----------------- ---------- annotated-types 0.6.0 anyio 4.2.0 certifi 2023.11.17 distro 1.9.0 exceptiongroup 1.2.0 h11 0.14.0 httpcore 1.0.2 httpx 0.26.0 idna 3.6 numpy 1.26.3 openai 1.6.1 pip 22.0.2 pydantic 2.5.3 pydantic_core 2.14.6 setuptools 59.6.0 sniffio 1.3.0 tqdm 4.66.1 typing_extensions 4.9.0
まずはテキスト埋め込みのみの実行。
compute_embeddings.py
import time from openai import OpenAI texts = [ "passage: Hello World.", "passage: こんにちは、世界。" ] start_time = time.perf_counter() openai = OpenAI(base_url="http://localhost:8080/v1", api_key="dummy-api-key") texts_embeddings = openai.embeddings.create(input=texts, model="intfloat-multilingual-e5-base") for text, data in zip(texts, texts_embeddings.data): print(f"Text: {text}") print(f"Embedding: {data.embedding}") print(f"Dimention: {len(data.embedding)}") print("") elapsed_time = time.perf_counter() - start_time print(f"elapsed time = {elapsed_time:.3f} sec")
実行。
こうなりました。
$ python3 compute_embeddings.py Text: passage: Hello World. Embedding: [0.017963348, 0.022452803, -0.008030383, 0.013427042, 0.0073604425, -0.01246953, -0.00800022, -0.02613945, 0.03894455, 0.03666252, -0.009627855, -0.023873258, 0.17872535, 0.03209413, -0.05457048, -0.059981775, -0.0019603956, -0.018795725, 0.03309854, 0.0003548984, 0.04086059, -0.029321175, 0.049620517, -0.035942532, 0.034320433, 0.003847589, 0.026393235, 0.012632506, -0.015205075, 0.03147812, 0.029287929, -0.014838075, 0.0033834935, 0.013282766, 0.02874948, 0.042601027, -0.009821255, -0.040816672, 0.05357897, 0.017244149, 0.017035251, 0.044196602, 0.006299907, -0.0060696877, 0.008316645, -0.00449509, 0.0116965575, 0.009954469, -0.02405501, -0.028371507, -0.005067132, 0.007995602, 0.03772363, 0.017130189, -0.05271493, -0.0636196, 0.025603332, 0.023617996, -0.04015092, 0.025899505, -0.023568667, 0.0087562185, -0.0037550412, 0.030815892, 0.024056124, -0.020960784, -0.0034925884, 0.0060334117, -0.03050062, -0.027349127, -0.0132043585, -0.029113017, 0.033056896, -0.04154473, -0.038592365, -0.039462518, -0.052376114, 0.021195993, -0.018792782, -0.0107929185, 0.028783562, -0.0024419613, 0.0486246, 0.04722122, -0.004763428, -0.0106682945, -0.0355246, 0.031859256, 0.026606563, 0.053462215, 0.014225056, 0.031875163, -0.04972084, 0.026375247, -0.00016270184, 0.00882226, 0.012693379, 0.0233597, 0.029646004, -0.050920133, 0.022015093, -0.022001361, -0.036716074, -0.026030004, -0.0681074, 0.001655258, -0.012927914, -0.002355498, 0.03415326, -0.011598739, -0.025342343, 0.01984709, 0.024940908, -0.046858724, 0.023859628, -0.025186155, 0.018016757, -0.030861514, -0.010432418, -0.051873874, -0.013479405, 0.05983331, -0.00982699, -0.017908162, 0.04475443, -0.018127605, -0.001092443, -0.006873188, 0.040993165, -0.023047553, 0.02595192, -0.03708721, -0.011978052, 0.0060347365, -0.046463598, 0.031469043, 0.037550073, -0.0025641457, -0.0050403047, 0.0024739127, -0.0013109507, -0.06748982, 0.03257076, 0.020918595, 0.037504617, -0.0561293, 0.042982146, 0.020988949, -0.026929097, 0.05583245, 0.027134672, -0.05353055, -0.014421977, -0.007159693, 0.03034546, -0.0063849837, -0.061118625, -0.020709815, 0.004795571, 0.030758947, 0.03969976, 0.026835999, 0.009739388, -0.011943362, 0.043690976, -0.002846591, -0.0024523272, 0.00084896554, 0.0018559992, -0.04877821, -0.014756407, 0.0028667815, -0.052282065, 0.0043575508, 0.013625516, -0.011973373, -0.0048611523, -0.041330498, -0.032809842, -0.06811807, -0.043061327, -0.049132794, -0.007237197, 0.03075986, -0.011706384, -0.034618776, -0.004317763, -0.008562017, 0.031101732, 0.000883508, -0.008786094, 0.05091839, 0.029521246, 0.025280938, 0.021518698, 0.037464894, 0.045456227, 0.036311038, -0.014685185, -0.045321934, 0.032041945, -0.015325965, 0.017751254, 0.03167474, -0.029392246, -0.06564554, 0.017786698, 0.04928254, 0.044800375, 0.030372038, -0.03536473, 0.017527845, -0.040722497, 0.024587972, -0.031679835, -0.041440126, 0.013890432, 0.002290624, -0.030883495, 0.04052506, 0.030804662, -0.06780065, 0.031926915, -0.011725815, 0.00827905, 0.022508912, 0.04858463, -0.019405209, 0.02332204, 0.021154596, 0.015630648, 0.029894995, -0.047343988, 0.022839205, -0.015833583, -0.030915882, 0.02446429, -0.0031016576, -0.038426388, -0.11470535, -0.012519229, 0.022111757, 0.020221606, -0.023698589, 0.028840147, -0.0551004, -0.027462954, 0.049345743, -0.04528969, 0.022106966, 0.018566478, -0.024529686, 0.019473884, -0.000858994, 0.022978734, 0.03091585, -0.05013637, 0.007873796, 0.00088438776, 0.00857614, -0.015608435, -0.04675861, -0.025120348, -0.0032357979, -0.003438804, 0.028345328, 0.10203295, -0.03661555, -0.008030549, -0.010248762, -0.03737399, -0.021812359, -0.07564361, 0.0434987, 0.02883957, -0.03470381, -0.08516729, 0.04837109, -0.028710369, -0.0031414968, -0.04123234, 0.05362493, -0.018825704, -0.054204438, -0.03232332, 0.031489298, -0.034656126, -0.029217005, -0.08352172, 0.027780337, 0.058643278, 0.0348726, -0.007324714, 0.03381329, 0.04100753, 0.010341152, 0.06637519, -0.00018712837, 0.013359329, 0.002475559, -0.022030674, -0.037783008, 0.018894665, -0.03167545, -0.041783873, 0.01473403, 0.10389064, -0.0015584624, -0.051446002, 0.06826191, -0.016788794, -0.03651429, -0.053504776, 0.022006588, 0.06199148, -0.047604326, -0.0041734315, 0.082690775, 0.01101869, 0.04672908, -0.034674257, 0.030564057, -0.045861628, -0.0657363, 0.033878144, 0.019803207, 0.05172596, -0.009832603, 0.0761512, 0.0066351616, -0.0040302244, 0.03491353, -0.026842427, 0.067995235, -0.09055586, -0.006781856, -0.015994286, -0.0065156505, 0.06643556, 0.0155476, -0.06942109, 0.035509665, -0.01003874, 0.001644982, 0.060444195, -0.03793358, 0.0057467977, -0.015630078, -0.023817347, 0.05659232, 0.031980585, -0.030771276, 0.03751321, 0.010623769, -0.034159143, -0.024705188, 0.053343456, -0.035094373, -0.018981935, -0.054727316, -0.03336836, 0.04154971, -0.004157062, -0.0032397748, 0.021941667, 0.03553393, -0.00052957615, -0.011035524, 0.042013027, 0.0027530382, 0.09791109, 0.0353259, -0.032781076, -7.969276e-05, 0.046401523, -0.041863926, 0.059148904, -0.026570547, -0.008279196, 0.043345954, 0.05527138, 0.04924404, 0.036653794, -0.00760489, -0.025631728, 0.040688474, 0.02362384, -0.062767915, 0.047478024, -0.031580806, 0.010884482, -0.03728399, -0.025193423, 0.01857102, 0.028072985, -0.028246636, 0.039823335, 0.0033638412, -0.003672612, -0.047882553, 0.024632076, 0.024762765, -0.012211261, 0.058566827, 0.03472563, 0.0054776804, 0.004005914, 0.044146646, -0.018141633, -0.024196686, 0.043984875, 0.016355967, 0.07082987, 0.018856723, -0.0043838318, -0.022748973, -0.018507572, 0.051205996, 0.04271425, 0.009626896, -0.032686472, 0.0436928, 0.024762394, 0.008027977, 0.020712644, -0.082668476, -0.026238572, -0.022668337, 0.011518707, -0.03729032, -0.024653131, 0.009901843, 0.004161004, -0.0064464128, 0.024507422, -0.05335617, 0.010474655, -0.014601111, -0.001887863, 0.011439716, -0.020642169, -0.01088923, -0.005896765, -0.018107736, -0.03704107, -0.0032512986, -0.02919667, -0.017159529, -0.015139074, -0.00804797, 0.022196159, 0.0046371403, 0.031038318, -0.0075136023, 0.018344354, -0.07134462, 0.050925326, -0.01376036, 0.003937521, -0.064963296, 0.05057103, 0.026968682, -0.02614072, -0.015127838, -0.008537493, -0.058274366, 0.023700928, -0.0109536275, 0.008866198, -0.021478364, -0.02240781, -0.040212784, 0.034757327, 0.020471402, 0.022780335, 0.019039098, -0.0049707983, 0.05871985, 0.031354893, -0.029321548, -0.010867838, -0.024946066, -0.036775358, 0.030215932, 0.042245578, 0.032155566, -0.068389006, 0.047944967, 0.017748581, -0.015451506, -0.009276643, 0.032235082, 0.046635713, -0.014784236, 0.010777993, 0.046848476, -0.010698888, -0.10616334, 0.027734803, 0.022742584, -0.03222744, -0.0030997603, -0.043010287, -0.002215993, -0.06528028, 0.04010143, -0.022578998, -0.010837071, 0.042060863, -0.03081479, 0.11970195, -0.063269354, -0.021328613, -0.051168203, 0.008616167, 0.0011367421, -0.0288364, 0.023756398, -0.0037965938, -0.028299717, -0.0018830964, -0.00092233584, -0.029832736, -0.015919749, 0.016221782, -0.03268509, 0.007620817, -0.04449706, 0.024154121, -0.028702253, 0.0402626, -0.055708542, 0.017017502, -0.08316179, -0.05797757, 0.021814527, -0.013021868, -0.02551646, 0.05798188, -0.018483363, -0.027480278, -0.05143044, 0.031820007, -0.03026568, 0.022808945, -0.03132677, 0.011753702, -0.007576147, -0.005234944, 0.00070803193, 0.024181519, -0.013321157, 0.019451443, 0.054206084, -0.030401127, 0.02046555, -0.019808007, 0.051773716, 0.017604792, -0.028957065, -0.015423858, 0.0019066518, 0.003104887, 0.013327669, -0.01430447, 0.011737523, 0.004828599, -0.018664693, -0.057049293, 0.07023137, -0.09850859, 0.018082177, 0.027977884, -0.0065359324, 0.06860984, 0.01790935, -0.024298305, -0.03524098, 0.04315675, -0.050641038, -0.025967123, -0.022571608, -0.025719251, 0.050323028, -0.008015487, 0.024477975, 0.037716717, -0.040037587, -0.21617962, 0.023267506, -0.018457353, -0.020526776, -0.004869364, -0.021799957, -0.017036254, 0.0028744412, 0.026463259, 0.02683716, 0.04426463, -0.09169242, 0.069441006, -0.0044798325, -0.013981939, 0.026078582, 0.0016064432, -0.04660366, 0.017603643, 0.038467668, 0.004514875, 0.055198062, -0.03928408, 0.0011049875, -0.04772851, -0.036018897, 0.0019565967, 0.0012631973, 0.011568942, 0.007095831, -0.018473329, 0.009743112, 0.043620966, 0.04569733, -0.023225678, 0.0010451281, -0.03517821, 0.0139313275, -0.084699675, 0.05262261, 0.009041638, -0.06416174, -0.021391813, 0.029345008, -0.053400915, 0.053389594, -0.0041594463, 0.009527862, -0.038559347, -0.0343955, 0.057667725, 0.030153992, -0.08978779, -0.017757144, -0.0370004, 0.052126057, 0.027457738, 0.0049967095, 0.05756858, -0.023213219, 0.055217102, 0.03414621, -0.019269723, 0.031927917, -0.018014377, -0.009899107, 0.015522806, -0.020175166, -0.012470183, 0.009669396, 0.0146638565, -0.009691908, -0.029672036, 0.034973405, 0.036494438, -0.0414563, 0.022893917, -0.022093058, -0.0711556, 0.018352985, -0.029981755, -0.06352266, 0.0054167314, 0.02020138, -0.027571449, -0.0012520222, 0.013499906, 0.0033101488, 0.04586074, 0.040114485, -0.004906896, -0.003921434, 0.025193883, 0.0038072474, 0.047968175, -0.020239944, -0.01827727, 0.052634545, 0.014220027, -0.018427968, -0.0122501, 0.0037733086, -0.0068801846, 0.032165997, 0.020551017, -0.024139626, -0.07129771, 0.027280606, -0.0051508443, -0.05749168, 0.014126195, 0.021527821, 0.048675075, -0.02330197, -0.013613943, -0.034286946, 0.014208377, -0.05033755, -0.06422418, -0.06681029, -0.024878042, -0.015350983, 0.028654022, 0.0024231374, 0.04147812, -0.0042125564, -0.012937028, 0.033775676, 0.026175462, 0.013504658, -0.007523545, -0.03279221, 0.08132918, -0.03516918, -0.0058922125, 0.08159558, 0.03838752, 4.3129465e-05, 0.004858068, 0.034548316, 0.061277125, -0.02513253, -0.002440704, 0.010181078, -0.051380213, 0.029111339, 0.036748316, -0.04292471, 0.037923712, 0.0035700344, -0.032362618, -0.061699174, -0.010605051, -0.019787407, -0.028379034, 0.004741903, 0.016848287, 0.014112461, -0.028533334, 0.028439645, -0.02057434, -0.04845035, 0.032774046, 0.027118472, 0.03248956, 0.012374039, -0.007939039, -0.010162315, 0.011687787, 0.018056182, -0.024776846, -0.012956437, 0.03566769, 0.0033593657, 0.01212312, -0.0064096344, 0.04650084, -0.056670852, -0.04442688, 0.028273867] Dimention: 768 Text: passage: こんにちは、世界。 Embedding: [0.018371977, 0.017757587, -0.01704158, 0.029186666, 0.011952086, 0.00011549716, -0.0039567235, -0.025779335, 0.029518101, 0.040523853, -0.0054477686, -0.041388318, 0.18205026, 0.043241117, -0.06161731, -0.019885978, -0.0042697294, -0.016027287, 0.021321207, 0.01293389, 0.043057542, -0.029955689, 0.052121688, -0.027150678, 0.046947703, -0.014890938, 0.010990443, -0.0024386551, -0.017234782, 0.029996185, 0.03361974, -0.01943105, 0.02922602, 0.019086732, 0.031039156, 0.04771883, -0.009413401, -0.030739466, 0.032999482, 0.008856295, 0.004699424, 0.053503003, 0.0070674047, -0.011284005, 0.018410893, -0.011264066, 0.02801581, 0.018295411, -0.026863039, -0.027039373, 0.0047116815, 0.031775657, 0.014787501, 0.00038495418, -0.040161375, -0.058688335, 0.029180277, 0.044417888, -0.063470885, 0.033244453, 0.008889456, 0.007974301, -0.008671685, 0.0060444716, 0.030493991, -0.03015665, 0.010165133, -0.006744991, -0.04291766, -0.0322743, 0.00925876, -0.01805136, 0.049208093, -0.038087767, -0.05320158, -0.036719486, -0.033270728, 0.022875706, -0.027117202, -0.010153142, 0.026746443, -0.009504743, 0.036088, 0.028231079, -0.014944845, -0.019397568, -0.05504761, 0.018071359, 0.011240429, 0.06330939, 0.026852569, 0.010765516, -0.032645885, 0.027197324, 0.0011513983, -0.0065323394, 0.027728805, 0.04119353, 0.03102409, -0.030630281, 0.001674493, -0.084030956, -0.025129806, -0.033577856, -0.09498135, -0.018296927, -0.0015294204, -0.021902146, 0.04796655, -0.011791072, -0.039243612, 0.039763365, 0.008758982, -0.071925744, 0.012390106, -0.015446737, 0.016008958, -0.024776766, -0.007389653, -0.051104505, 0.006831203, 0.037859827, -0.017845884, -0.016355049, 0.043631703, -0.0012695004, 0.006837076, -0.018034108, 0.046924774, -0.033651464, 0.0048500886, -0.044487756, -0.0059658885, 0.018640626, -0.020907816, 0.05913745, 0.03363274, 0.009713657, 0.0138532, -0.0064184405, -0.003917931, -0.07417509, 0.010565746, 0.0346258, 0.047628336, -0.04405678, 0.03421438, 0.015162379, -0.026760217, 0.044546776, 0.050577603, -0.06229815, -0.021927742, -0.019564241, 0.02594959, 0.02108712, -0.050920255, -0.019083124, 0.009909659, 0.046323758, 0.0332278, 0.0321207, 0.014691952, -0.0139727425, 0.020925773, -0.008043842, -0.0024820126, 0.014254057, -0.003913434, -0.04828036, 0.002051171, 0.0141305765, -0.0558916, 0.01604602, 0.016042218, -0.00022930263, -0.02283773, -0.040868096, -0.04424666, -0.06919518, -0.059542377, -0.045316055, -0.029455982, 0.034692705, -0.01580027, -0.024961865, -0.004874856, -0.017832339, 0.012271005, 0.0055327187, -0.009742657, 0.073738396, 0.012996689, 0.029568637, 0.030648733, 0.039457235, 0.025243733, 0.024957746, 0.007265382, -0.06864609, 0.036253784, -0.029856712, 0.027822815, 0.026369963, -0.03743069, -0.07402901, 0.003971899, 0.076708354, 0.03208938, 0.032175053, -0.020198401, 0.026510943, -0.015482409, 0.011572637, -0.005064185, -0.04803742, 0.042546824, 0.0044366266, -0.025749413, 0.054296937, 0.053682383, -0.0637415, 0.06925102, -0.0040422566, 0.0026814332, 0.038690444, 0.03645585, -0.032746404, 0.009047295, 0.018193632, 0.007927477, 0.024819124, -0.042426463, 0.050561126, -0.039148927, -0.021480242, 0.0081841955, 0.0041575707, -0.04250609, -0.12455038, -0.005033593, 0.0012320204, 0.011257233, -0.02257404, 0.037730377, -0.034340132, -0.027363924, 0.036701497, -0.035666697, 0.01668758, 0.011260227, 0.0026107482, 0.00579025, 0.008855892, 0.0035524115, 0.024828993, -0.046780948, 0.02706487, -0.0030975894, 0.0012935328, -0.013379203, -0.029397234, -0.037661366, -0.012681427, -0.0054361178, 0.019908978, 0.06730965, -0.055053778, -0.033724956, -0.020787856, -0.018480463, -0.020353224, -0.05390102, 0.045913797, 0.020236187, -0.019767871, -0.079856314, 0.043118183, -0.009945355, -0.006372752, -0.031725813, 0.05408863, -0.0324469, -0.04703178, -0.035944045, 0.02018422, -0.037127715, 0.013140075, -0.083154686, 0.022934888, 0.04776724, 0.059493914, 0.002625536, 0.049386233, 0.038281366, 0.030646272, 0.06110163, 0.0028329573, 0.030298168, 0.006858534, -0.03083938, -0.025916658, 0.022621918, -0.016544798, -0.020884207, 0.0029841417, 0.12109001, 0.006777061, -0.054969, 0.05181593, -0.0024484168, -0.029717475, -0.04416181, 0.038782556, 0.05008325, -0.04974734, 0.015854442, 0.084120065, 0.021743536, 0.030534204, -0.013394751, 0.045422446, -0.028134989, -0.0436387, 0.058428522, -0.0026635346, 0.043951962, -0.028346233, 0.05582784, 0.022361562, -0.024009522, 0.013763814, -0.035429392, 0.05894187, -0.070215344, 0.0011875876, -0.014294846, -0.0017284058, 0.08312878, 0.0006565213, -0.08684755, 0.012371725, 0.009934104, -0.012519865, 0.048644245, -0.06324156, -0.0080158, -0.017145822, -0.027314475, 0.030516164, 0.04383933, -0.06223654, 0.040841606, 0.011063589, 0.0007915946, -0.008129988, 0.025261123, -0.029012496, -0.020631818, -0.029306509, -0.03750366, 0.041851822, 0.0114627, 0.0054780147, 0.028813548, 0.036673095, -0.004186186, 0.00904267, 0.034065694, -0.0125754, 0.08899043, 0.046076536, -0.04040277, -0.0019169605, 0.05498343, -0.036583614, 0.044933945, -0.048003502, -0.01913324, 0.040959228, 0.07475859, 0.044653606, 0.017151682, -0.005127077, -0.024198141, 0.031282242, 0.04958256, -0.06961782, 0.05123001, -0.017383771, 0.020031437, -0.020939033, -0.024205444, 0.018373586, 0.026135264, -0.023729058, 0.04108824, -0.032741453, 0.0010041405, -0.031355027, 0.02494227, 0.015193209, 0.009938055, 0.040109176, 0.012666854, -0.021165127, 0.0019867455, 0.03315809, -0.015532589, -0.040482774, 0.051835332, 0.0053902664, 0.07808579, 0.015729649, 0.006793204, -0.03382878, -0.017357135, 0.061035864, 0.038513973, 0.002613007, -0.037953127, 0.03746798, 0.011478838, 0.013032557, 0.026187494, -0.088162474, -0.027242292, -0.020876227, 0.013897377, -0.030429563, -0.038053997, 0.008236474, 0.028820988, -0.028611083, 0.026993722, -0.05499197, 0.021499725, -0.017846586, 0.0062417695, 0.015614782, -0.004910819, -0.005454367, -0.011509174, -0.026087288, -0.03745234, -0.0033196877, -0.052234657, -0.016741453, -0.03082185, -0.000947639, 0.029738786, 0.005320358, 0.032281026, -0.009351219, 0.028683772, -0.060605202, 0.04863498, -0.021472316, 0.0023940606, -0.09713149, 0.046230797, 0.025261307, -0.028136311, -0.009690455, -0.018794166, -0.05275874, 0.032079592, -0.034535173, -0.004709103, -0.029939465, -0.051345684, -0.032406192, 0.009992332, 0.010079649, 0.04511393, 0.0049833083, 3.1613238e-06, 0.05429155, 0.033154782, -0.045027528, 0.0038643943, -0.018867943, -0.050321158, 0.033001967, 0.0058742007, 0.013595602, -0.06592714, 0.08414038, 0.041085806, -0.02994323, -0.0023035477, 0.036883973, 0.019657109, -0.011733486, 0.0052131554, 0.06153752, -0.031092655, -0.0694157, 0.043284513, 0.0063573453, -0.030650483, -0.009210659, -0.04172123, -0.014264078, -0.06364828, 0.026323773, -0.030347202, -0.00023547647, 0.040290527, -0.03141451, 0.064786084, -0.06667221, -0.03539293, -0.05545319, 0.02035005, -0.0022307143, -0.038806733, 0.030650858, -0.0011296349, -0.022809332, -0.014320531, 0.010665389, -0.0434239, -0.029351322, 0.024542224, -0.03784304, 0.017423548, -0.060242314, 0.02694023, -0.0011230914, 0.028161407, -0.038921352, 0.010561092, -0.052193988, -0.081754684, 0.021136919, -0.029431157, -0.034830365, 0.028875565, -0.026296264, -0.03577052, -0.047633387, 0.009805114, -0.026793307, 0.022591677, -0.03353921, 0.03567524, -0.009558187, -0.007922776, -0.0013149341, 0.0016373215, -0.0040863245, -0.007755242, 0.04364897, -0.053081725, -0.0017859174, 0.0049721124, 0.05319598, 0.01236362, -0.017167153, 0.014884835, 0.00051266793, -0.0035774384, 0.017265523, 0.0022465577, 0.01578847, 0.0066322195, -0.014754986, -0.032250196, 0.08333298, -0.101475015, 0.005565663, 0.03161856, -0.019290194, 0.03655528, 0.038991787, -0.0061464217, -0.029668437, 0.03614795, -0.04807168, -0.03319626, 0.010287311, -0.027731426, 0.045047794, 0.025763143, 0.01694869, 0.04002134, -0.016332028, -0.19511242, 0.02876407, -0.0133339865, -0.017760899, -0.0025103863, 0.013732659, -0.007337027, 0.0026081596, 0.02288278, 0.017315911, 0.046604343, -0.06849307, 0.03179692, 0.012790093, -0.023127066, 0.023948913, -0.008326839, -0.057167653, 0.019582035, 0.041118953, 0.002684436, 0.06750929, -0.018933397, -0.002890965, -0.05221182, -0.027866976, -0.031667, 0.020233184, 0.015721813, 0.0138778305, -0.022690225, 0.017167894, 0.012254423, 0.045696344, -0.034487274, -0.0018696024, -0.03345154, 0.030956408, -0.025603488, 0.054561857, -0.010850044, -0.061873443, -0.025789747, 0.026994998, -0.027692093, 0.06860284, -0.0026781633, -0.010603914, -0.03183464, -0.0069039604, 0.044620834, 0.026964396, -0.122409835, -0.021471737, -0.035973776, 0.044359006, 0.03496921, 0.024721833, 0.05322031, -0.020870456, 0.05425467, 0.03249944, -0.010224002, 0.026591409, -0.011743992, -0.029483246, 0.025485862, -0.013681688, -0.009631198, -0.0017953267, 0.03184719, -0.011172662, -0.04478992, 0.04336509, 0.022692971, -0.031236231, 0.01802404, 0.00024216388, -0.07900698, 0.026945427, -0.012786574, -0.072703786, -0.023390446, 0.017781302, -0.02014207, 0.0023414178, 0.008701411, -5.063182e-05, 0.03699452, 0.05044223, -0.015826467, 0.0032796073, 0.014697565, -0.014150072, 0.035693932, -0.020656683, -0.018699257, 0.04941772, 0.00909162, -0.04107555, 0.0011824161, 5.067929e-05, -0.0020818508, 0.033732086, 0.030707989, -0.04055341, -0.07343493, 0.0452523, -0.025879838, -0.07070902, 0.037148807, 0.019203275, 0.028491061, -0.020298867, 0.01644141, -0.041522834, 0.0048005534, -0.033458173, -0.048993755, -0.05609739, -0.03449305, -0.022801923, 0.028969413, -0.0094586685, 0.03557929, 0.017522365, -0.030419959, 0.025009643, 0.018667273, 0.054159734, -0.0150813805, -0.041563984, 0.057728283, -0.039473556, -0.01970022, 0.039468803, 0.030084772, -0.029246962, 0.0005132708, 0.031786796, 0.05915461, -0.058531504, -0.025335325, 0.0057240142, -0.03179796, 0.01581596, 0.0540899, -0.028469004, 0.05302103, -0.012695831, -0.013914782, -0.06775897, -0.012215999, -0.021440908, -0.028012997, -0.0033265932, 0.017419936, 0.020876758, -0.019965835, 0.036580887, -0.0103459265, 0.010969579, 0.033980045, 0.01751057, 0.024093535, 0.024514, -0.013699262, -0.0062247426, 0.017820593, 0.013666421, -0.017565984, -0.010688422, 0.043934032, 0.009703778, 0.033255372, -0.00090386404, 0.02763535, -0.049361657, -0.03613376, 0.039794594] Dimention: 768 elapsed time = 0.273 sec
ひとまず、動きましたと。
続いては、テキストの類似度を使用して簡単な検索をしてみます。
SentenceTransformersを直接使って書いた前のエントリーのソースコードを、OpenAI Python APIライブラリーを使ったものに
書き換えました。
text_similarity_search.py
import sys import time from openai import OpenAI import numpy as np ## https://github.com/openai/openai-python/blob/v0.28.1/openai/embeddings_utils.py#L65-L66 def cosine_similarity(a, b): return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)) start_time = time.perf_counter() documents = [ "passage: 特急に乗っています。", "passage: 今から実家へ帰ります。", "passage: 九州へ行きます。", "passage: リンゴを食べます。", "passage: 釣りに行ってきます。", "passage: 魚を食べます。", "passage: 肉を食べます。", "passage: みかんが欲しいです。", "passage: セーターを着ます。", "passage: コートを着ます。" ] openai = OpenAI(base_url="http://localhost:8080/v1", api_key="dummy-api-key") embeddings = openai.embeddings.create(input=documents, model="intfloat-multilingual-e5-base") documents_with_embedding = [{ "document": document, "embedding": data.embedding } for document, data in zip(documents, embeddings.data)] query = f"query: {sys.argv[1]}" query_embedding = openai.embeddings.create(input=query, model="intfloat-multilingual-e5-base").data[0].embedding documents_with_similarity = [{ "document": d["document"], "embedding": d["embedding"], "similarity": cosine_similarity(query_embedding, d["embedding"]) } for d in documents_with_embedding] sorted_documents = sorted(documents_with_similarity, key=lambda d: d["similarity"], reverse=True) print("ranking:") for document in sorted_documents: print(f" document: {document['document']}") print(f" similarity: {document['similarity']:.3f}") print() elapsed_time = time.perf_counter() - start_time print(f"elapsed time = {elapsed_time:.3f} sec")
確認してみます。
$ python3 text_similarity_search.py 帰省する ranking: document: passage: 今から実家へ帰ります。 similarity: 0.849 document: passage: 九州へ行きます。 similarity: 0.829 document: passage: 釣りに行ってきます。 similarity: 0.815 document: passage: セーターを着ます。 similarity: 0.807 document: passage: 特急に乗っています。 similarity: 0.807 document: passage: みかんが欲しいです。 similarity: 0.806 document: passage: 肉を食べます。 similarity: 0.801 document: passage: コートを着ます。 similarity: 0.798 document: passage: 魚を食べます。 similarity: 0.795 document: passage: リンゴを食べます。 similarity: 0.783 elapsed time = 0.574 sec $ python3 text_similarity_search.py 食事 ranking: document: passage: 肉を食べます。 similarity: 0.868 document: passage: 魚を食べます。 similarity: 0.856 document: passage: リンゴを食べます。 similarity: 0.833 document: passage: みかんが欲しいです。 similarity: 0.820 document: passage: 釣りに行ってきます。 similarity: 0.814 document: passage: 九州へ行きます。 similarity: 0.814 document: passage: 今から実家へ帰ります。 similarity: 0.809 document: passage: 特急に乗っています。 similarity: 0.803 document: passage: セーターを着ます。 similarity: 0.802 document: passage: コートを着ます。 similarity: 0.793 elapsed time = 0.611 sec $ python3 text_similarity_search.py 寒い ranking: document: passage: セーターを着ます。 similarity: 0.815 document: passage: コートを着ます。 similarity: 0.814 document: passage: みかんが欲しいです。 similarity: 0.814 document: passage: 特急に乗っています。 similarity: 0.813 document: passage: 九州へ行きます。 similarity: 0.810 document: passage: 今から実家へ帰ります。 similarity: 0.807 document: passage: 釣りに行ってきます。 similarity: 0.807 document: passage: 肉を食べます。 similarity: 0.806 document: passage: 魚を食べます。 similarity: 0.795 document: passage: リンゴを食べます。 similarity: 0.782 elapsed time = 0.583 sec
結果はSentenceTransformersを使った時と、コサイン類似度を含めて同じになっているので大丈夫そうですね。
これで試したかったことは確認できました。
まとめ
LocalAIのテキスト埋め込みのバックエンドにSentenceTransformersを使うようにしてみました。
動いてしまえば使うのは簡単なのですが、環境を作るのにとても苦労しました…。
自分はDockerイメージ版を使うよりもまずインストールしてみて感覚を掴もうとすることが多いのですが、これはちょっと厳しかった
ですね…。バージョンアップの度にこれをやるのはあんまり考えたくないです（笑）。
ただ、この構成ができたので、SentenceTransformersを使ったテキスト埋め込みをOpenAI API経由で実行できるようになるのは便利ですね。
こちらは後に使っていくことになると思います。
