Code generation with Kaggle Models

For code generation, we will experiment with the Code Llama model, the 13b version. From the LLMs available on the Kaggle platform at the time of writing, this model was the most appropriate, in regards to its purpose (it is a model specialized for code generation) and size (i.e., we can use it with Kaggle Notebooks), for the task of code generation. The notebook used to demonstrate the code generation is given in Reference 9. The model is loaded, quantized using bitsandbytes, and has a tokenizer initialized in the same way, as demonstrated in Reference 7. We define a prompt and a pipeline (using the transformers function) with the following code:

prompt = 'Write the code for a function to compute the area of circle.'
sequences = pipeline(
    prompt,
    do_sample=True,
    top_k=10,
    temperature=0.1,
    top_p=0.95,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
    max_length=200,
)

The result of executing...