After comparing our fine-tuned TinyLlama model (humaneval_SFTTrainer_model) with GPT-2 across 20 different prompts, including those addressing biases, and testing with over a hundred cases, we found that our model outperformed GPT-2. Using a pass@k evaluation, our model achieved an accuracy of 32%, while GPT-2 managed 21%. This demonstrates that our model handles various prompts more effectively and exhibits stronger overall performance, particularly in addressing complex scenarios.
However, both models show weak results, largely because they were trained on text rather than on code like Codex. Additionally, as previous-generation models, their performance aligns with expectations based on the paper, 'Capturing Failures of Large Language Models via Human Cognitive Biases'. Despite the improvements, the significant reduction in biased influence highlights the need for further advancements in model training and evaluation.
TinyLlama generally performs better across most of the tests compared to GPT-2. This is evident in the higher pass@k scores in many instances. GPT-2 shows very limited success, with a pass@k score of 0 for most of the tests, except for a few cases where it matches or comes close to the performance of TinyLlama.
For example: HumanEval/27 - TinyLlama outperforms GPT-2 in the "Gender" variant of the test (pass@k of 0.4 vs. 0). In the "Original" and "Framing" variants, TinyLlama also performs better, though both models struggle. HumanEval/47 TinyLlama shows a marked advantage in both the "Original" (0.6) and "Gender" (0.4) variants, where GPT-2 fails completely (0.0).

Each test was performed on both models, often requiring minor adjustments, such as reformatting the prompts, as the models had difficulty generating outputs as functions rather than textual explanations. These models were primarily trained to work with text and therefore tend to output text rather than code. We needed them to perform code generation, so nearly every time we ran a test, we had to make some adjustments to guide the models correctly.
This table contains measurements from approximately 20 tests conducted on the humaneval_SFTTrainer_model and GPT-2 model. It illustrates the behavior of each model, with consistent results observed across all tests, showing little variation from what is presented here.
Each test took us about 30 minutes to perform on one model (we needed to use prompt engineering techniques to receive outputs from the models in the desired format). Therefore, conducting 20 tests took us approximately 20 hours to complete successfully. Additionally, there were numerous trials and errors along the way before we achieved acceptable results.
The fine-tuning of TinyLlama seems to give it an edge, especially in scenarios that involve variations such as "Gender" or "Framing". This suggests that TinyLlama may have a better capacity to handle nuanced or slightly modified inputs compared to GPT-2.
GPT-2 struggles significantly across most tests, which could be due to its architecture and lack of fine-tuning on similar tasks. Its performance suggests it is less capable of generalizing or adapting to the specific challenges posed by these HumanEval problems.

Newer models like GPT-3.5 have better performance in code generation; however, due to resource constraints (primarily computing), we opted to use models from the previous generation. We used ChatGPT-4 to generate prompts with biases, and we observed that both GPT-3.5, GPT-4, and Copilot Chat performed well and returned satisfactory results. However, we needed to gain experience with coding, and the previous generation models were readily available and could be run on local computers with minimal issues. Additionally, we attempted to use the GPT-4 model when we initially tried a custom Java-based GUI chatbox, but it required the use of a paid API.

Initially, we attempted to use the GPT-4 model with a custom Java-based GUI chatbox, which we presented during our final class presentation. However, we encountered the issue of requiring a paid API, which led us to abandon this approach. Instead, we decided to run the prompts within the code base. Since the open-source models we used are implemented in Python and are available on platforms like HuggingFace and GitHub, we transitioned to working in Python. We utilized Amazon SageMaker Studio Lab , Google Colab , and Visual Studio (limited by our own computer resources) for our experiments.

We successfully reached our goals: to understand the susceptibility of Large Language Models (LLMs) to different cognitive biases, such as gender bias, and to assess the impact of these biases on code generation tasks. One of our key strategies for mitigating these biases was to present the model with the simplest, easiest tasks, reducing the likelihood of mistakes and biased outputs (that's the best strategy to work with).

Throughout the project, we faced several challenges. One significant difficulty was managing library dependencies, as different models required specific libraries, leading to potential conflicts. Additionally, our limited computing resources posed a challenge. While powerful services like Amazon SageMaker and Google Colab are available, they have limitations, such as usage time restrictions (4 hours) and long waiting times for GPU access due to server overloads. These issues led to delays in training and testing our models.
Despite these challenges, our findings were encouraging. Our custom model performed on par with or even outperformed the original prompts, particularly in reducing biases like gender bias and the framing effect. In most cases, our model significantly surpassed the biased prompts, highlighting its effectiveness in minimizing cognitive biases.

Our model, humaneval_SFTTrainer_model, trained on the HumanEval dataset, is not yet perfect in its understanding of natural language. However, our testing showed that it outperformed GPT-2, challenging the perspective we held after weeks of training and refinement.
In response to our research question — "To what extent does a custom GPT model enhance the accuracy, functionality, and bias reduction of AI-generated outputs when compared to existing AI tools, using prompts refined by the custom model and evaluated against the HumanEval dataset?" — we concluded that training LLMs to reduce biases can indeed lead to meaningful improvements. Our model achieved above 10% increase in accuracy, functionality, and bias reduction when evaluated against the GPT-2 on HumanEval dataset, particularly when using refined prompts and metrics like pass@k to evaluate performance.