如何使用Codex获取令牌或代码嵌入？

Question

对于给定的代码片段，如何使用Codex进行嵌入？

import os
import openai
import config


openai.api_key = config.OPENAI_API_KEY

def runSomeCode():
    response = openai.Completion.create(
      engine="code-davinci-001",
      prompt="\"\"\"\n1. Get a reputable free news api\n2. Make a request to the api for the latest news stories\n\"\"\"",
      temperature=0,
      max_tokens=1500,
      top_p=1,
      frequency_penalty=0,
      presence_penalty=0)

    if 'choices' in response:
        x = response['choices']
        if len(x) > 0:
            return x[0]['text']
        else:
            return ''
    else:
        return ''



answer = runSomeCode()
print(answer)

但是我想找出像下面这样的python代码块，我能从codex获得嵌入吗？

输入：

import Random
a = random.randint(1,12)
b = random.randint(1,12)
for i in range(10):
    question = "What is "+a+" x "+b+"? "
    answer = input(question)
    if answer = a*b
        print (Well done!)
    else:
        print("No.")

输出：

输入代码

• 嵌入

Answer 1

函数get_embedding将为输入文本提供嵌入。

这里来自OpenAI的规范代码：https://github.com/openai/openai-python/blob/main/examples/embeddings/Get_embeddings.ipynb

import openai
from tenacity import retry, wait_random_exponential, stop_after_attempt

@retry(wait=wait_random_exponential(min=1, max=20), stop=stop_after_attempt(6))
def get_embedding(text: str, engine="text-similarity-davinci-001") -> List[float]:

    # replace newlines, which can negatively affect performance.
    text = text.replace("\n", " ")

    return openai.Embedding.create(input=[text], engine=engine)["data"][0]["embedding"]

embedding = get_embedding("Sample query text goes here", engine="text-search-ada-query-001")
print(len(embedding))

Answer 2

是的，OpenAI可以为任何输入文本创建嵌入--即使它是代码。您只需要在其get_embedding()函数调用中传递正确的引擎或模型。我测试了这段代码：

# Third-party imports
import openai

from openai.embeddings_utils import get_embedding


openai.api_key = OPENAI_SEC_KEY


embedding = get_embedding("""
    def sample_code():
        print("Hello from IamAshKS !!!")
""", engine="code-search-babbage-code-001")

print()
print(f"{embedding=}")
print(f"{len(embedding)=}")

# OUTPUT:
# embedding=[-0.007094269152730703, 0.006055716425180435, -0.005044757854193449, ...]
# len(embedding)=2048


embedding = get_embedding("""
import Random
a = random.randint(1,12)
b = random.randint(1,12)
for i in range(10):
    question = "What is "+a+" x "+b+"? "
    answer = input(question)
    if answer = a*b
        print (Well done!)
    else:
        print("No.")
""", engine="code-search-babbage-code-001")

print()
print(f"{embedding=}")
print(f"{len(embedding)=}")

# OUTPUT:
# embedding=[-0.011341490782797337, -0.005919027142226696, 0.0011923711281269789, ...]
# len(embedding)=2048

注意：您可以使用engine参数替换get_embedding()模型或引擎。

上面给出的代码可以为任何代码嵌入。还有一个名为code-search-ada-code-001的代码搜索引擎/模型，但它的功能不如我用来回答这个问题的code-search-babbage-code-001那么强大。如果您还想进行代码搜索，请参阅下面的引用。

参考资料：

• https://openai.com/blog/introducing-text-and-code-embeddings/
• https://beta.openai.com/docs/guides/embeddings/code-search-embeddings
• https://beta.openai.com/docs/guides/embeddings/use-cases#expander-7