How do you even compare search engines? For the longest time, this question was out of the question (sorry for the pun š„) because Google dominated this category so hard it wasnāt even worth asking.
Since the LLM boom, when it was clear that asking ChatGPT questions in natural language is much preferable to contorting oneself to create the best search query, folks started seeing the potential end to Googleās reign of the place that answers your questions.
Heck, even Google in the last Google IO said they are working on a new feature that will ālet Google do the Googling for youā
Iāve been a long time user of Perplexity Pro and honestly I love it, Iāve set it up as my default search and use it quite often, and the latest research mode update was really helpful for me to research a family vacation to Dubrovnik, discover day trip requirements and much more!
So I was quite surprised and excited to see that OpenAI announced SearchGPT, their temporary prototype of a reimagined AI based search and QUICKLY signed up to the waitlist, because, well, Iām into testing AI things, so here we go, hereās a quick comparison between the two AI powered search engines that you all asked for š
* This is to acknowledge that Perplexity is a mature product that has an amazing team and SearchGPT is a temporary prototype from a company that does TONS of other stuff
Review approach
While the video above was recorded in 1 shot and you can see me running through a few examples, in the post below I will try to review and compare both search engines on the following criteria (you can consider this the table of contents for this post)
UI & UX
Real Time Search Results
Local results
Complex & Multistep queries
Multilinguality
Widgets & Share-ability
(there are definitely more ways to compare and evaluate, please feel free to leave comments as to what else I should test with SearchGPT)
UI & User Experience
The first thing I Immediately noticed was the super sleek UI, there are barely any elements on the page, you type and you get an answer. Thereās also a typeahead suggestion as you type, which Perplexity doesnāt have (yet, they apparently just launched it in beta to some customers)
On the downside, thereās no search history tabs, no settings, no knobs to tweak.
The main User Experience wow effect comes from the speed, SearchGPT is FAST, it feels faster than Perplexity, and while itās not faster than Google, Google just gives me links (at least for the queries I tried, did they completely remove their UI after the embarrassing responses with the pizza glue incident?) and SearchGPT answers complex queries super quick which matters a LOT for user experience.
SearchGPT also has a header that summarizes the answer in one sentence, which is very helpful to quickly glance at. On the left sidebar there are a few buttons for a list of sources for the current search and images. And thatās basically eat.
Iāll give this category to Perplexity for now, just because itās a fleshed out product, has a mobile app, has follow up questions, and just feels more mature, but I really like the simplicity of SearchGPT!
Searching for stuff that JUST happened
The first thing I wanted to test is how up to date SearchGPT is compared to Perplexity, given that OpenAI just launched it (but has a Bing backing) itās interesting to see how fast they are indexing things that just happened
What is the price of Bitcoin?
This one was difficult to even test, because well, both got it wrong, while both claimed they had live data (which none really did, CoinGecko showed a different price) and bitcoin fluctuates quite a LOT.
Does Alex Volkov host a course for Weights & Biases?
Friday, my teammate Agata surprised me by saying she used a bunch of my videos and stitched them together into a course about our product Weave (which you can see here btw). Since I know that this page went live yesterday, I wanted to see how fast both these search engines index that new page.
Both fail to find it, but whatās really interesting is, during the recording of the video above (which took place 5 hours before Iām writing this), Perplexity DID have the right answer and did find the right course with the right citation (so did google) so I donāt know why did Perplexity decide to forget that I have this course after 5 hours but this is not looking good for PPXL :/
When does the olympics ceremony start?
This one was easy, but thereās a trick here, I first looked this up with Perplexity before the ceremony started, but now I looked it up, while the ceremony already passed, and here, SearchGPT is the clear winner. Why? Well, because both PPXL and Google told me when the ceremony is GOING to be, while SearchGPT correctly surmised that it has already taken place, which is a way more natural way to talk about things that happened!
Not only that, it looks like SearchGPT pulls the eventās photograph (since it already happened!) which makes for a much nicer result
What time did it air where I live?
One of the best things about these new AI searches is the ability to follow up and dig deeper into an answer, and weirdly, even tho I switched to GPT-4 Omni in PPXL, the same exact follow up failed to give me the answer I wanted in PPXL and worked in SearchGPT.
When I tested again, and phrased a bit better (What time did it air where I live?), this worked in PPXL as well š¤·āāļø Weird, given supposedly itās the same model underneath? Or maybe OpenAI is running a new model under there??
What was the last episode of ThursdAI about?
This failed in both search engines, while SearchGPT gave an outdated answer, citing a post of mine from June 13 (over a month and a half ago), PPXL just gave me a vague generic answer with no details and no citings and no dates š¤
Is this a Substack issue? I tried multiple queries and couldnāt get either search to lead me to the episode I released yesterday š®āšØ At least neither of these try to correct my question to Thursday like Google does š
Local Results
Both search engines know where I live based on IP, so letās see how they handle local stuff.
Is it going to rain tomorrow?
Huh? Why does PPXL think Iām in NY here? I was 100% sure that previously it knew where I was? It took me a few seconds to notice that I had the āPro searchā mode one, which apparently puts me in NY!
Once I repeated the search without Pro mode, PPXL found out about where I live, and gave me a nice weather widget, but also was more correct. According to my verification with Apple weather, thereās 0% of rain tomorrow.
Iāll give this one to Perplexity! SearchGPT made me carry an āļø for nothing! (who am I kidding, this is Denver, itās always in the car!)
Weather for the next week
Now that both services found where I am, asking for a weather report for the whole week, shows that SearchGPT has a nice weekly weather widget, very clear and clean, while PPXL, what, I donāt even š¤¦āāļø The graph is still there, but the description says it has no data!
Complex and Multistep queries
This is the area where LLM based searches supposed to shine compared to a regular 10 links based Google like search, especially once they do some advanced processing in the background. Here I will say, Perplexity has a āproā mode that I absolutely love, that takes longer, breaks down our search into multiple queries and then responds with a great summary, but that takes time so I donāt often do that. For sake of comparison I will try with it both on and off.
What's more expensive, the DMNS or Childrens Museum family plus annual package?
DMNS is Denver Museum of Nature and Science, this is an actual question I wanted answers to, requires visiting two websites, reading and doing some comparisons.
Weirdly here, SearchGPT went lazy on me and didnāt give me the Childrenās Museum annual price this time, while it absolutely did give it to me while recording the video š¤ and answered my question correctly. This shows that both PPXL and SearchGPT often non deterministic results to the same exact queries within a span of around 5 hours! Itās not a shock to me given how LLMs work, but still something to note.
Winner here : Perplexity
What will be the temperature in Hawaii when it'll be the first day of school in Denver?
I donāt know how to judge this one, I think both are wrong-ish. While PPXL did give the correct date of start of school year, SearchGPT just estimated it (again, in the video it estimated correctly, here in screenshot it did NOT), PPXL then proceeded to estimate the weather while SearchGPT tried to at least go and find the weather during this date.
Maybe this one was too complex and needs something like the Pro mode in PPXL š¤
How to make the best Shakshuka in the world?
I much prefer the way SearchGPT presents recipes TBH, they are extracting an image that fits the exact step they are describing and itās really cool. Perplexity shows instructions in a very detailed way.
I will fail both search engines here as someone who knows Shakshuka, both recommended canned tomatoes, and for the BEST shakshuka, you need fresh ones as everyone knows š
But UI wise, I think SearchGPT takes this one
Multilinguality
LLMs can speak a bunch of languages, letās see if the search engines can also handle queries and results in different languages
What happened in the opening of the Olympic games (in Hebrew)
Both searches handled this query well, with PPXL giving way more details. One thing to mention also, PPXL shows the Hebrew in RTL (Right to Left) aligned format, while SearchGPT doesnāt.
PPXL gets the win here 100%
How much faster is it to fly to Croatia from Denver than to Hawaii (Russian)
This one is a tricky question, because itās not faster, flying to Croatia is actually slower, but I wanted to see how the answers will be, PPXL takes this one, as there are no direct flights between Denver and Croatia which it rightly notes!
Neither has issues with the Russian here.
Widgets and Share-ability
I havenāt discovered all the widgets that both PPXL and SearchGPT have, so hard to compare, and itās obvious that the more mature product will have tons more, but they at least have multi media and images showing for some results, weather related widgets and more
The one thing I like about SearchGPT is the highlighted short answer at the top, while PPXL repeats the query twice (you can see it at the top and then again as the main header on the page), SearchGPT shows your query in grey and the answer in black
PPXL has an amazing shareability option, which I absolutely love, the social media cards are incredible, sharing with friends is a delight, not to mention turning search results into pages. Meanwhile, SearchGPT results cannot be shared (makes sense given this is a limited gated preview)
Should I put glue on pizza?
I had to try and see if the infamous pizza glue incident will happen with these and nope, not only do both of these search engines warn me that itās not the right thing to do, SearchGPT even throws shade at Google š
In Summary:
SearchGPT is a great addition to the chatGPT family of products and features, itās not easy to build a search engine, and even harder to build one from scratch, and even double harder to build one that doesnāt lead to embarrassing examples that will eventually make your company to roll back these features!
This is quite an incredible release from OpenAI given that this isnāt the main focus of their company, and while itās lacking in features still, and sometimes hallucinates, itās already very useful, super fast and quite awesome to use, and even at this early stage has some advantages over established competitors!
Will I be switching away form PPXL? Maybe for a bit, I like to test drive things to the fullest, but Iāll probably wonāt be using this as my default search for long, I LOVE PPXL PRO š«”
Hope you enjoyed this deep dive, see you next ThursdAI
