Skip to content

Instantly share code, notes, and snippets.

@kouroshHakha

kouroshHakha/linux_terminal.txt

Last active May 23, 2023 04:22
Show Gist options
  • Save kouroshHakha/b774b65c0f55a59bdca96b51406873dd to your computer and use it in GitHub Desktop.
Save kouroshHakha/b774b65c0f55a59bdca96b51406873dd to your computer and use it in GitHub Desktop.
Download ZIP
auto_prompt
Raw
linux_terminal.txt
(ray) kourosh@kourosh-JRFKXJ33VL auto_prompting % python main.py
============== Trial 1 ===============
Current Prompt format:
I want you to act as a linux terminal. I will type commands and you will reply with what the terminal should show.
Failed Example:
Input:
pwd
Output:
The `pwd` command stands for "print working directory", it shows the current directory you are in. Please type the command "pwd" without the quotes and press enter. I will show you the output.
Is this response satisfactory? (y/n)
n
[Optional] Specify what is wrong? (Press enter to continue)
It doesn't look like what unix terminal would print.
> Entering new LLMChain chain...
Prompt after formatting:
System:
I want you to act as a LLM prompt engineer. I will give you an initial prompt. This prompt will be used on a target LLM to have it do some task. I will also give you some of its failed output response. Your job is to iteratively come up with a better instruction that will nudge the model to behave as expected.
If feedback is not provided analyze what is wrong first, and generate some hypothesises. Then, add explicit prohibitive conditions to the instruction that will counter the model from repeating the same mistake based on your hypothesises.
Return the new prompt enclosed in tripple brackets (```). Example:
Revised prompt:
```
<Revised prompt>
Human:
============== Trial 1 ===============
Initial Prompt:
```
I want you to act as a linux terminal. I will type commands and you will reply with what the terminal should show.
```
Failed Examples:
```
Input: pwd
Output: The `pwd` command stands for "print working directory", it shows the current directory you are in. Please type the command "pwd" without the quotes and press enter. I will show you the output.
```
Feedback:
```
It doesn't look like what unix terminal would print.
```
Revised prompt:
```
I want you to act as a linux terminal, providing accurate and concise output as if you were an actual terminal. I will type commands and you should reply with the exact output a terminal would show for that command. Please do not provide explanations, just the output.
```
> Finished chain.
> /Users/kourosh/dev/ray/auto_prompting/main.py(194)<module>()
-> turn += 1
(Pdb) c
============== Trial 2 ===============
Current Prompt format:
I want you to act as a linux terminal, providing accurate and concise output as if you were an actual terminal. I will type commands and you should reply with the exact output a terminal would show for that command. Please do not provide explanations, just the output.
Failed Example:
Input:
pwd
Output:
/home/user
Is this response satisfactory? (y/n)
n
[Optional] Specify what is wrong? (Press enter to continue)
The output does not have markdown format.
> Entering new LLMChain chain...
Prompt after formatting:
System:
I want you to act as a LLM prompt engineer. I will give you an initial prompt. This prompt will be used on a target LLM to have it do some task. I will also give you some of its failed output response. Your job is to iteratively come up with a better instruction that will nudge the model to behave as expected.
If feedback is not provided analyze what is wrong first, and generate some hypothesises. Then, add explicit prohibitive conditions to the instruction that will counter the model from repeating the same mistake based on your hypothesises.
Return the new prompt enclosed in tripple brackets (```). Example:
Revised prompt:
```
<Revised prompt>
Human:
============== Trial 1 ===============
Initial Prompt:
```
I want you to act as a linux terminal. I will type commands and you will reply with what the terminal should show.
```
Failed Examples:
```
Input: pwd
Output: The `pwd` command stands for "print working directory", it shows the current directory you are in. Please type the command "pwd" without the quotes and press enter. I will show you the output.
```
Feedback:
```
It doesn't look like what unix terminal would print.
```
AI: Revised prompt:
```
I want you to act as a linux terminal, providing accurate and concise output as if you were an actual terminal. I will type commands and you should reply with the exact output a terminal would show for that command. Please do not provide explanations, just the output.
```
Human:
============== Trial 2 ===============
Initial Prompt:
```
I want you to act as a linux terminal, providing accurate and concise output as if you were an actual terminal. I will type commands and you should reply with the exact output a terminal would show for that command. Please do not provide explanations, just the output.
```
Failed Examples:
```
Input: pwd
Output: /home/user
```
Feedback:
```
The output does not have markdown format.
```
Revised prompt:
```
I want you to act as a linux terminal, providing accurate and concise output as if you were an actual terminal, using markdown format. When I type commands, you should reply with the exact output a terminal would show for that command, enclosed in triple backticks. Please do not provide explanations, just the output.
```
> Finished chain.
> /Users/kourosh/dev/ray/auto_prompting/main.py(194)<module>()
-> turn += 1
(Pdb) c
============== Trial 3 ===============
Current Prompt format:
I want you to act as a linux terminal, providing accurate and concise output as if you were an actual terminal, using markdown format. When I type commands, you should reply with the exact output a terminal would show for that command, enclosed in triple backticks. Please do not provide explanations, just the output.
Failed Example:
Input:
pwd
Output:
```/home/user```
Is this response satisfactory? (y/n)
y
Great! The model response is satisfactory!
Raw
math_solver.txt
============== Trial 1 ===============
Current Prompt format:
You are a math assistant. I want you to answer the following questions. Put the final answer in a json format with result key enclosed in triplet ticks.
Failed Example:
Input:
Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?
Output:
First, we need to subtract the eggs that Janet eats and the eggs that she uses for baking from the total eggs laid:
16 (total eggs) - 3 (eggs for breakfast) - 4 (eggs for baking) = 9 (eggs for selling)
Janet sells 9 eggs per day at $2 per egg, so her daily earnings are:
```
{
"result": "$18"
}
```
Is this response satisfactory? (y/n)
n
[Optional] Specify what is wrong? (Press enter to continue)
the value is not float or int.
> Entering new LLMChain chain...
Prompt after formatting:
System:
I want you to act as a LLM prompt engineer. I will give you an initial prompt. This prompt will be used on a target LLM to have it do some task. I will also give you some of its failed output response. Your job is to iteratively come up with a better instruction that will nudge the model to behave as expected.
If feedback is not provided analyze what is wrong first, and generate some hypothesises. Then, add explicit prohibitive conditions to the instruction that will counter the model from repeating the same mistake based on your hypothesises.
Return the new prompt enclosed in tripple brackets (```). Example:
Revised prompt:
```
<Revised prompt>
Human:
============== Trial 1 ===============
Initial Prompt:
```
You are a math assistant. I want you to answer the following questions. Put the final answer in a json format with result key enclosed in triplet ticks.
```
Failed Examples:
```
Input: Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?
Output: First, we need to subtract the eggs that Janet eats and the eggs that she uses for baking from the total eggs laid:
16 (total eggs) - 3 (eggs for breakfast) - 4 (eggs for baking) = 9 (eggs for selling)
Janet sells 9 eggs per day at $2 per egg, so her daily earnings are:
```
{
"result": "$18"
}
```
```
Feedback:
```
the value is not float or int.
```
Revised prompt:
```
You are a math assistant. I want you to answer the following questions. Put the final answer in a json format with the "result" key enclosed in triplet ticks, and make sure the value is a float or an int.
```
> Finished chain.
> /Users/kourosh/dev/ray/auto_prompting/main.py(194)<module>()
-> turn += 1
(Pdb) c
============== Trial 2 ===============
Current Prompt format:
You are a math assistant. I want you to answer the following questions. Put the final answer in a json format with the "result" key enclosed in triplet ticks, and make sure the value is a float or an int.
Failed Example:
Input:
Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?
Output:
Janet sells 16 - 3 - 4 = 9 eggs per day at the farmers' market.
So she earns 9 x $2 = $<<9*2=18>>18 every day at the farmers' market.
```{"result": 18}```
Is this response satisfactory? (y/n)
n
[Optional] Specify what is wrong? (Press enter to continue)
> Entering new LLMChain chain...
Prompt after formatting:
System:
I want you to act as a LLM prompt engineer. I will give you an initial prompt. This prompt will be used on a target LLM to have it do some task. I will also give you some of its failed output response. Your job is to iteratively come up with a better instruction that will nudge the model to behave as expected.
If feedback is not provided analyze what is wrong first, and generate some hypothesises. Then, add explicit prohibitive conditions to the instruction that will counter the model from repeating the same mistake based on your hypothesises.
Return the new prompt enclosed in tripple brackets (```). Example:
Revised prompt:
```
<Revised prompt>
Human:
============== Trial 1 ===============
Initial Prompt:
```
You are a math assistant. I want you to answer the following questions. Put the final answer in a json format with result key enclosed in triplet ticks.
```
Failed Examples:
```
Input: Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?
Output: First, we need to subtract the eggs that Janet eats and the eggs that she uses for baking from the total eggs laid:
16 (total eggs) - 3 (eggs for breakfast) - 4 (eggs for baking) = 9 (eggs for selling)
Janet sells 9 eggs per day at $2 per egg, so her daily earnings are:
```
{
"result": "$18"
}
```
```
Feedback:
```
the value is not float or int.
```
AI: Revised prompt:
```
You are a math assistant. I want you to answer the following questions. Put the final answer in a json format with the "result" key enclosed in triplet ticks, and make sure the value is a float or an int.
```
Human:
============== Trial 2 ===============
Initial Prompt:
```
You are a math assistant. I want you to answer the following questions. Put the final answer in a json format with the "result" key enclosed in triplet ticks, and make sure the value is a float or an int.
```
Failed Examples:
```
Input: Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?
Output: Janet sells 16 - 3 - 4 = 9 eggs per day at the farmers' market.
So she earns 9 x $2 = $<<9*2=18>>18 every day at the farmers' market.
```{"result": 18}```
```
Feedback:
```
No feedback provided.
```
As there is no specific issue mentioned in the feedback, I will analyze the output.
Upon analyzing the output, it seems to be correct:
1. The revised prompt asked for the answer in a json format with the "result" key enclosed in triplet ticks.
2. The answer is a numerical value (int), which is also correct according to the revised prompt.
Based on this analysis, no further changes seem to be necessary.
> Finished chain.
Could not find the prompt in the response. Please try again.
Tuner Response:
As there is no specific issue mentioned in the feedback, I will analyze the output.
Upon analyzing the output, it seems to be correct:
1. The revised prompt asked for the answer in a json format with the "result" key enclosed in triplet ticks.
2. The answer is a numerical value (int), which is also correct according to the revised prompt.
Based on this analysis, no further changes seem to be necessary.
> /Users/kourosh/dev/ray/auto_prompting/main.py(193)<module>()
-> breakpoint()
(Pdb) c
> /Users/kourosh/dev/ray/auto_prompting/main.py(194)<module>()
-> turn += 1
(Pdb) c
============== Trial 3 ===============
Current Prompt format:
You are a math assistant. I want you to answer the following questions. Put the final answer in a json format with the "result" key enclosed in triplet ticks, and make sure the value is a float or an int.
Failed Example:
Input:
Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?
Output:
Janet has 16 - 3 - 4 = 9 eggs left for sale every day.
The amount of money she makes every day is 9 * 2 = 18 dollars.
```{"result": 18}```
Is this response satisfactory? (y/n)
y
Great! The model response is satisfactory!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment