Text Generational models

  • OpenAi’s text generation models, like GPT-4 have been trained to understand natual and formal language.
  • The inpus to these models are refered to as prompts. Desiginign a prompt is basically how you program a model like GPT-4, suually by providing instructions or some examples of how to siccesfully complete a task
  • to use oneof the OpenAI models API, you should sen da request containgin the inputs and your ApI key, and recieve a response containing the models output.
  • gpt-4 and gpt- 3.5 can be accessed through the chat completions API endpoint
    • API endpoint for newer models: https://api.openai.com/v1/chat/completions

example CHat Completions API call:

from openai import OpenAI
client = OpenAI()

response = client.chat.completions.create(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Who won the world series in 2020?"},
    {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
    {"role": "user", "content": "Where was it played?"}
  ]
)
  • the ain input is the messages prameters.

  • Messages must be an array of messages objects, where each object has a role( either” system”, “user”, or “assistant”)

  • Conversation is usually formatted withh a system message first followed by alternatig user and assistant messages

  • The system message helps set the behavior of the assistant. For example, you can modify the personality of the assistant or provide specific instructions about how it should behave throughout the conversation.

  • Including conversation history is important when user instructions refer to prior messages.
    • In the example above, the user’s final question of “Where was it played?” only makes sense in the context of the prior messages about the World Series of 2020. Because the models have no memory of past requests, all relevant information must be supplied as part of the conversation history in each request.

An example Chat Completions API response

{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "The 2020 World Series was played in Texas at Globe Life Field in Arlington.",
        "role": "assistant"
      },
      "logprobs": null
    }
  ],
  "created": 1677664795,
  "id": "chatcmpl-7QyqpwdfhqwajicIEznoc6Q47XAyW",
  "model": "gpt-3.5-turbo-0613",
  "object": "chat.completion",
  "usage": {
    "completion_tokens": 17,
    "prompt_tokens": 57,
    "total_tokens": 74
  }
}

assistants reply can be extracted with:

response['choices'][0]['message']['content']
  • Every response will include a finish_reason. The possible values for finish_reason are:
    • stop: API returned complete message, or a message terminated by one of the stop sequences provided via the stop parameter
    • length: Incomplete model output due to max_tokens parameter or token limit
    • function_call: The model decided to call a function
    • content_filter: Omitted content due to a flag from our content filters
    • null: API response still in progress or incomplete

What are tokens

  • Language models read and write text in chunks called tokens. In English, a token can be as short as one character or as long as one word (e.g., a or apple), and in some languages tokens can be even shorter than one character or even longer than one word.
    • For example, the string “ChatGPT is great!” is encoded into six tokens: [“Chat”, “G”, “PT”, “ is”, “ great”, “!”].
  • The total number of tokens in an API call affects:
    • How much your API call costs, as you pay per token
    • How long your API call takes, as writing more tokens takes more time
    • Whether your API call works at all, as total tokens must be below the model’s maximum limit (4097 tokens for gpt-3.5-turbo)

How to build the chatbot in python

  • install pip3 install open AI
  • install the official Node.js library, run ‘npm install open ai@^^4.a.a in Node.js project directory
  • get and api key from website, make an account and API key

Secret API key that I generated: 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

import openai

openai.api_key = "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"

def chat_with_gpt(prompt):
    response = openai.Completion.create(
        model= "gpt-3.5-turbo",
        essages= [{"role": "user", "content": prompt}]
    )
    return response.choices[0].messages.content.strip

if __name__ == '__main__':
    while True:
        user_input = input('You: ')
        if user_input.lower() in ["quit", "exit", "bye"]:
            break 
        response = chat_with_gpt(user_input)
        print("Chatbot: ", response)

---------------------------------------------------------------------------

APIRemovedInV1                            Traceback (most recent call last)

Cell In[10], line 17
     15 if user_input.lower() in ["quit", "exit", "bye"]:
     16     break 
---> 17 response = chat_with_gpt(user_input)
     18 print("Chatbot: ", response)


Cell In[10], line 6, in chat_with_gpt(prompt)
      5 def chat_with_gpt(prompt):
----> 6     response = openai.Completion.create(
      7         model= "gpt-3.5-turbo",
      8         essages= [{"role": "user", "content": prompt}]
      9     )
     10     return response.choices[0].messages.content.strip


File /opt/homebrew/lib/python3.11/site-packages/openai/lib/_old_api.py:39, in APIRemovedInV1Proxy.__call__(self, *_args, **_kwargs)
     38 def __call__(self, *_args: Any, **_kwargs: Any) -> Any:
---> 39     raise APIRemovedInV1(symbol=self._symbol)


APIRemovedInV1: 

You tried to access openai.Completion, but this is no longer supported in openai>=1.0.0 - see the README at https://github.com/openai/openai-python for the API.

You can run `openai migrate` to automatically upgrade your codebase to use the 1.0.0 interface. 

Alternatively, you can pin your installation to the old version, e.g. `pip install openai==0.28`

A detailed migration guide is available here: https://github.com/openai/openai-python/discussions/742