Yes, the tech is impressive, even if it doesn’t always generate factually correct responses. The weird thing about this in software development communities online, especially groups focused on supporting new developers, is the examples of it being used are where new developers are using the tech to help them find examples to answer ‘how do I…?’ or ‘show me an example of …?’ type questions. The generated responses can be mostly correct, with text generated from source material that the model has been trained on, but the shocking realization of these examples shared online is that you could have found exactly the same content if you Googled for it.
This is why Google is worried. Not that they don’t have a comparable product readily available right now. They’re worried that traditional search traffic and therefore ad revenue suddenly has an alternative, one that is gaining a lot of interest and hype, and maybe for the first time in years, suddenly there is a threat that search traffic that would have previously gone to Google is now going to go somewhere else.
Microsoft’s announcement yesterday that they are adding ChatGPT integration into their Bing search engine hits the nail on the head. They didn’t announce a page where you can go and ask weird questions, they’re building it in to their search engine.
There’s something fundamentally game changing to the search (and ad venue) )industry about this. Instead of searching for key words and phrases like we’ve all got used to now for years, being able to ask a vague question on a topic and get what you’re looking for is a game changer. Instead of searching for links to content on other websites that have been indexed, you can now search knowledge and ask questions in a conversational style to find the information you’re looking for. That is a game changer.
I’m setting up an approach to run text generation model training jobs on demand with aitextgen, and the first approach I’m looking at is to run the training in a Docker container. Later I may move this to an AWS service like ECS, but this is my first step.
I’ve built a Docker image with the following dockerfile:
RUN yum update -y
RUN yum install -y python3
RUN pip3 install aitextgen
ADD source-file-for-fine-tuning.txt .
ADD generate.py .
ADD train.py .
.. and then built my image with:
docker build -t aitextgen .
I then run a container passing in the cmd I want to run, in this case ‘python3 train.py’:
docker run --volume /data/trained_model:/trained_model:rw -d aitextgen sh -c "cd / && python3 train.py && mv aitextgen.tokenizer.json /trained_model"
I’m also attaching a bind point where the model output is being written to during the run, and -d to run the container in the background. The last step in the run command copies the token file to the mounted EBS volume so it can be reused by the generation.
To generate text from the model, run:
docker run --volume /data/trained_model:/trained_model:rw -d aitextgen sh -c "cd / && python3 generate.py"
And then providing a prompt of something relevant like :
“The problem with training Machine Learning models today is that the model is only as good as the data it is trained with.”
and let’s see what we get:
This almost sounds believable as if it was written by hand. It might not make complete sense, but the sentences are well formed, something that was an issue with my previous experimenting with RNN model text generation.
Here’s. the text for reference:
“There is a desire to train the model on a large data set.
A very big data set is not always a good enough data set.
A good data set is more than just enough. A good data set is always growing.
To take advantage of this growing data set we need to train the model on the latest batch of data. However, there may be a huge amount of data that does not fit in the pre-defined batch size.”
I’m curious how the temperature setting effects what is generated, so let’s bump it up from 0.7 to 0.9 and try again with the same prompt text.
This sounds like it comes straight out of a research paper:
“This is why there is now a trend toward training a workforce of highly trained humans in a search engine optimization function (SRE) to monitor changes in signal and value in the big data.
Whether it is a search engine algorithm like Chi-squared or a machine learning model like a neural network, there is still a lot of work ahead to make those apps more relevant to the data they collect.
The need for better data
Having data that makes the predictions bigger, on”
Not knowing exactly how this works, it almost reads like it’s regurgitating whole chunks of the data it was trained on, but who knows. Given that it was trained from materials and sites online, I guess the easy way to test for this would be to just drop one of these sentences in Google and see if something like the source article comes up. Trying this out:
Huh. So this sentence is not found anywhere in Google’s indexes. That’s quite remarkable if the model actually generates sentences that are this convincing.
What about starting with a line from a recipe (for example from here):
“Heat oil in large pot. Add beef and flour, stirring to coat beef; brown.”
Ok, now this is interesting but slightly weird. It’s clearly recognizable as steps in a recipe, but the order doesn’t make too much sense and some of the steps are repeated variations of themselves:
“Add onions and garlic and cook until onion starts to turn translucent, 3 minutes. Stir in bell pepper, glass tomatoes, and celery; cook 1 minute. Add potatoes, beef broth, and bay leaves; bring to a boil. Reduce heat to low. Reduce heat to medium; cook until beef is no longer pink. Taste and add salt and pepper. Add beef and stir. Add liquid; cover and simmer until meat is tender, 10 minutes.”
What about lines from films?
“I find your lack of faith disturbing”
Now we’re gone off into Crazy Town. This is weird:
“Why do you persist in viewing my appearance with contempt?”
“I want to believe I am worthy of your worship.”
“Therefore, you have no right to demand I lift my gaze.”
“I am unworthy of your reverence; I can only pray that you may address me in a less lewd manner.”
“You call me a beast?”
“You cannot deny the very fact you see me.”
“I see nothing.”
Whoah. Ok, keeping with the Star Wars theme, what does it do if you start it with some Yoda quotes:
“When 900 years old you reach, look as good you will not”
And I got:
“I listen to every guru and I say, good luck with your yoga and counting numbers.”
Does this phrase exist anywhere?
Nope. You have to wonder if there were some Joda quotes or lines from films that went into the model, it’s interesting that ‘yoga’ is mentioned, and you wonder if this is the model generating a variation of the name ‘Yoda’. And ‘When 900 years old you reach” = “Counting numbers”? Very curious.
Ok, one more. How about a line from an HP monitor product description:
“Designed with purpose: multiple customization options allow you to adjust your monitor to best fit the way you work.”
“Customizable paired ergonomics and colors: customize every feature of your new monitor from its basic design and color customization options.
Intuitive control links: there are more than 100
cabling connections provided with each monitor.
Designed for comfort: more than twice as many inches of space between the screen and the desk top than standard monitors.
Dedicated power buttons for powering the monitor on and off.”
Ok, enough playing, I did have a project planned to use with this, so time to get busy 🙂