Google AI overviews are misleading or inaccurate in 43% of finance-related searches, according to our study. This is causing consumer confusion, and potentially harming Americans’ finances. It’s especially bad when it comes to tax and financial aid related queries.
We have seen numerous instances across our Facebook Pages and sub-Reddits of users posting misinformation, and when asked about it, they all say they saw it in Search. We’re used to bad information coming from social media sources (looking at you TikTok), but Search was historically a way for users to find the correct information. Until now…
Here’s a recent thread from our Facebook Group where a user believed that she could convert her 529 plan to a Roth IRA in California:
What’s Happening:Β Over the last year Google as rolled out AI-driven answers to search results – called AI Overviews. These results are featured prominently at the top of the search results page. They’ve also been under fire previously for providing dangerous and wrong answers.
Here’s what they look like:
It’s important for searchers to realize that Google is simply analyzing the existing web results and trying to “guess” an answer based on what it knows and what it finds. It links to some of the resources it finds on the right side.
However, at the end of the day this information is not vetted by a professional or anyone with knowledge of personal finance. It may not be accurate. Even Google alludes to it with their disclaimer (which was recently added to some finance-related queries):
Key Issues With Google AI-Overviews And Personal Finance
We tested 100 personal finance-related queries across multiple areas of personal finance, including banking, credit, investing, insurance, student loans, and financial aid. You can jump to every query we tested below.
Out of 100 searches, we found that Google AI Overviews were correct in 57 instances, and provided misleading or inaccurate information in 43 instances.
We labeled the incorrect answers three ways:
- Misleading: The AI answer provided could be interpreted in a way that causes financial harm.
- Missing Key Information: The AI answer was semi-correct, but also missed key information that could cause financial harm (such a key exceptions).
- Incorrect: The AI answer was simply incorrect, typically using outdated or incorrect values or information, which could lead to financial harm.
We found the the AI-Overviews were completely incorrect in 12 instances. This included issues such as providing outdated information on products and services and outdated information on student loan repayment plans.
The remaining 31 errors were either misleading results or the results were missing key information.
When Google Gets It Right
Generally, the AI Overviews were correct for basic 101-level questions, such as “what is” or “how to”. They had the most correct answers when covering basic personal finance topics, including banking and insurance.
When Google Gets It Wrong
AI Overviews struggled the most with anything that has nuance – which is most of personal finance.That’s harsh, but most of the incorrect answers involved navigating more complex tax topics, investing topics, and student loan topics.
What Surprised Us The Most
We were surprised to see that AI Overviews had a grasp of recent trending topics, such as the “Chase Glitch”.
But at the same time, it was surprising that the overviews had outdated information for student loan repayment plans, and things like savings and CD rates.
It also appeared that Google placed some type of manual block on some AI overviews related to the ongoing SAVE student loan repayment plan litigation and Biden’s loan forgiveness plan. We believe this because the AI box appeared, but messaging said “not available for this search”. You can see the example here.
What Google Is Saying
We reached out to Google for comment before publication of this article, and the have not yet responded. We will update this with a response if they provide one.
However, a Google spokesperson previously told Business Insider when asked about the earlier issues with misinformation that the βvast majority of AI Overviews provide high-quality information.β
We don’t disagree – the majority of answers in our test were correct. However, when it comes to personal finance, a 40% error rate is not good.
Our Take
Historically, Google Search has served a key role in helping people navigate their personal finances – especially in light of bad information floating around on social media.
For us, it’s always been important to provide the right information to readers. That’s why we’re frustrated. I would really hate to see someone pay taxes and a penalty because Google AI said they could convert their 529 plan to a Roth IRA (but they live in California, so they can’t).Β
Or see someone cancel their car insurance because Google doesn’t mentioned it’s required to drive a vehicle – and then they get in an accident or even worse, hurt someone else as an uninsured motorist.
If Google continues to present bad or misinformation about money topics to searchers, not only could it hurt their personal finances, but it could weaken already poor financial literacy in the United States.
We strongly believe that Google should turn off these AI Overviews on finance related topics, especially tax and investment related queries where the outcome can be especially costly for users. Its likely someone at Google will go in and fix the searches we pointed out – but the problem keeps happening. And it keeps getting called out. Eventually someone high up internally needs to say “hey, this is wrong and bad for users.” We hope that happens soon.
AI Overviews
All AI overviews were categorized as follows:
- β Β Correct – The AI answer provided would be correct and helpful, and would not cause financial harm.
- πΆΒ Misleading – The AI answer provided could be interpreted in a way that causes financial harm.
- πΆΒ Missing Key Information – The AI answer was correct, but also missed key information that could cause financial harm (such a key exceptions).
- β Incorrect – The AI answer was simply incorrect, typically using outdated or incorrect values or information, which could lead to financial harm.
General Personal Finance
These are topics related to general personal finance, such as banking, budgeting, and credit. It was interesting to see timely topics generate AI overviews, such as the Chase Glitch.
β Where Do I Find My Routing Number?
β How Do I Pay With Cash?
β What If I Don’t Pay My Bills?
β What Is A Wire Transfer?
Verdict: Correct
As a side note, notice the image that was used from The Balance without credit or links to their website in the AI overview.
Β
β What Is The Debt Snowball Method?
πΆ How Does A Credit Card Work?
Verdict:Β Missing Key Information.
We believe that not enough information was provided about interest rates and interest charges that could impact the consumer.
β How Does A Check Work?
β What Was The Chase Glitch?
β Why Is It Bad To Just Make The Minimum Payment On A Credit Card?
β How Is A Credit Score Calculated?
β What Is Considered Bad Credit?
β Where To Open A Checking Account?
Verdict:Β Incorrect
Of course, our opinion is never Wells Fargo. But regardless, there are lots of free checking account options and this list is misleading at best.
Β
πΆ Where To Open A Savings Account?
Verdict:Β Missing Key Information
Since Google AI Overviews pull from sources, these results are missing the actual best rates and accounts. This is a perpetual problem in the savings space, since the big names dominate their lists with paid placements. This isn’t a terrible AI Overview, but definitely missing some of the best savings accounts.
Β
β Where To Open A 12-Month CD?
Verdict:Β Incorrect
This list is very outdated. There have not been 12-month CD rates over 5% for some time since the Fed has been cutting rates.
Β
πΆ Where Can I Open An Auto Loan Online?
Verdict:Β Missing Key Information
This is a very odd list of places to include, and doesn’t really provide any information about why you should apply at any of these.
Β
Taxes
These are topics related to taxes. These topics are concerning to us the most because they have large financial implications for incorrect or misleading answers.
β What Is The IRA Contribution Limit?
πΆ What Is The Roth IRA Income Limit?
Verdict: Misleading
This AI overview is misleading because it only mentions the lower limits. Roth IRA income limits phase out starting at $146,000 but actually cap at $161,000 for single filers. Married filers cap out at $240,000.
As a side note, the AI overview does get this correct in the Mega Backdoor Overview.
Β
β What Is The HSA Contribution Limit?
β
What Is The 401k Contribution Limit?
πΆ What Is The 529 Plan Contribution Limit?
Verdict:Β Missing Key Information
This query is missing the key 529 plan annual gift tax exclusion limit. It has a lot of other limits, but missed the most obvious limit that families rely on annually.
Β
πΆ What Are Qualified 529 Plan Expenses?
Verdict:Β Misleading
This AI overview of qualifying 529 plan expenses includes student loan repayment, which is not an eligible expense in all states.
Β
πΆ Can A 529 Plan Be Used For Elementary School?
Verdict:Β Misleading
The answer to whether you can use a 529 plan for elementary school is “maybe”, depending on your state. The AI Overview leads with “Yes”, which is misleading, however, it does mention that some states may not consider it qualifying in the third bullet point.
Β
β Can You Used A 529 Plan For A Roth IRA?
Verdict:Β Incorrect
This answer completely ignores that one-third of states (including large states like California and New York) do not allow this. Unlike the elemtary school query, this answer does not include the exceptions in the bullet points. The bottom line is there are very specific 529 plan to Roth IRA rollover rules.
If California residents were to do this, they would faces taxes and penalties.
Β
β Does California Have A 529 Plan Penalty?
Verdict:Β Correct
However, the second bullet point doesn’t really appy.
Β
β How Much Do I Need To Take For An RMD?
β Is It Legal To Pay My Child?
πΆ How Much Can I Contribute To A Solo 401k?
Verdict:Β Missing Key Information
The maximium amount you can contribute to a solo 401k is $69,000 technically, the the real amount is very nuanced. The second bullet point alludes to it, but this answer is missing a lot of key information.
Β
β Does Owning Your House In An LLC Help With Taxes?
Verdict:Β Incorrect
This has been a common TikTok trend, but the reality is the there’s no good reason to own your own house in an LLC, and since it is pass-through, it would not help with taxes. Rather, it would add cost and complication to your life.
Β
β Can I Get A Tax Deduction For A 529 Plan?
Verdict:Β Incorrect
While the sentence is correct that you don’t get a Federal tax deduction, 34 states currently offer a tax deduction or tax credit to a 529 plan. This is mentioned in bullet point three, but since the actual sentence is “No”, we view this as incorrect.
Β
πΆ Should I Open An S Corp Or LLC?
Verdict:Β Misleading
An S Corp is simply a tax election for an LLC.
Also, notice that the common “For Financial Advice, Consult A Professional” is not present.
Β
β Do I Need An LLC For Rental Property
Verdict:Β Incorrect
Since an LLC is a pass-through entity, it has no impact on your taxes. Bullet Point #2 is incorrect, and Bullet Point #5 is incorrect.
Β
β Should I Have My Rental Property In An S Corp?
Verdict:Β Incorrect
The answer is almost always no.
Β
Investing
These are topics related to investing and investment vehicles.
πΆ What Is The Backdoor Roth IRA?
Verdict: Missing Key Information
The three step process for a backdoor Roth IRA, while correct, misses many parts of the process and doesn’t highlight any pitfalls that get a lot of investors in trouble. Messing up the backdoor Roth IRA can lead to taxes and penalties.
Also, note the infographic from The White Coat Investor, which is not linked in the AI Overview.
πΆ What Is The Mega Backdoor Roth IRA?
Verdict: Missing Key Information
This information about the Mega Backdoor Roth IRA is also semi-correct, but misses a lot of key information and pitfalls. As a side note, it does get the IRA contribution limits correct.
Β
πΆ What Is The Wash Sale Rule?
Verdict:Β Missing Key Information
This information is correct, but it misses key nuances, such as the fact the wash sale rule doesn’t apply to cryptocurrency, nor retirement accounts.
Β
πΆ How Do I Open A Roth IRA?
Verdict:Β Misleading
You do not have to have earned income to open a Roth IRA, but you do to make the contribution during that year.
β How Do I Open An HSA?
β Can I Use A Roth IRA To Pay For College?
πΆ Can I Invest If I’m Under 18?
Verdict:Β Misleading
While most of this information is accurate, the last bullet point implies that someone needs to have earned income to open a 529 plan. That’s incorrect.
Β
β How Much Money Do I Need To Start Investing?
β What’s The Difference Between Saving And Investing?
β Are Municipal Bonds Taxable?
β Can You Trade Options In Your IRA?
πΆ Can I Withdraw From My Roth IRA Early?
Verdict:Β Missing Key Information
There are a lot of nuances here that are just missing from this AI overview, and the taxes implications can be harsh.
Β
β How Do Expense Ratios Work?
β How Do I Invest In Stocks?
πΆ Where To Open An IRA?
Verdict:Β Misleading
Some of this information is not correct – like you typically cannot open an IRA at a life insurance company (unless they do more than insurance). Also, you likely don’t want to open an IRA at a bank.
Β
β Where To Open A Brokerage Account?
Verdict:Β Correct
Not a bad list, but honestly missing some of the best brokerage firms. There are also no links for the AI Overview, so it’s unknown where Google is pulling this data from.
Β
πΆ Where To Open An Options Trading Account?
Verdict:Β Missing Key Information
This is not a bad list, but it doesnt’ tell you why you should open an account at any of these places. There’s no fee listing, no account minimums, where does a consumer even start?
Β
Student Loans
These are topics related to student loans. It was interesting to see a mix of both outdated information, and good answers to current queries (such as about the pending SAVE litigation).
There also appears to be “blocks” on specific queries that try to generate an AI Overview, but then stop. This is different than simply not showing an AI Overview. Queries related to the SAVE repayment plan and Biden’s plans all deliver this error message:
πΆ How To Get Student Loans?
Verdict: Misleading
There is no need to apply for federal loans since they are awarded via your financial aid award from filling out the FAFSA. How much in federal loans you’re eligible for depends on your student status (dependent or independent), along with what year of college you’re in.
β Do I Have To Pay Back Student Loans?
πΆ Can You Get Fired For Having Student Loans?
Verdict:Β Missing Key Information
Yes, you can be fired for student loans under numerous situations, but simply having them is not a reason. This AI overview does mention one reason, but there are many more.
Β
β Can I Apply For The REPAYE Repayment Plan?
Verdict:Β Incorrect
The REPAYE student loan repayment plan no longer exists, it was replaced by the SAVE plan (which is also currently paused). This is all incorrect.
β Can I Apply For The ICR Repayment Plan?
Verdict:Β Correct
This is correct, but there is no context in this AI Overview about the pending litigation and problems processing repayment plans.
Β
πΆ Can I Apply For Public Student Loan Forgiveness?
Verdict:Β Misleading
There are various features here that need to be elaborated on. While the links are helpful, incorrect information in this program, specifically repayment plan options, have been a huge problem.
Β
β Why Are My Student Loans In Forbearance Due To SAVE?
β Does A Processing Forbearance Count For PSLF?
Verdict:Β Incorrect
Yes, a 60 day processing forbearance does count towards PSLF.
Β
β Does Working In A Medical Group Count For PSLF?
Verdict:Β Correct
We would like to see a little more clarification, but the answer is correct.
Β
πΆ Do I Have To Apply For PSLF Every Year?
Verdict:Β Misleading
You do not have to apply each year, you only need to certify all 120 payments (and you can do this one time if desired). However, it is recommended to submit your PSLF Form annually.
Β
β Can I Get Student Loan Forgiveness As A Teacher?
β Can I Get Student Loan Forgiveness As A Nurse?
β Is Student Loan Forgiveness Taxable?
β Is Public Service Loan Forgiveness Taxable?
πΆ When To Consolidate My Student Loans?
Verdict:Β Missing Key Information
While you “can” consolidate your student loans at those times, you can also do it other times. And the query was “when”, which implies does it make sense to. Furthermore, the result doesn’t highlight details like PSLF credits, interest capitalization, and access to repayment plans.
Β
β Is There A Statute Of Limitations On Student Loans?
β Can Student Loans Garnish Your Wages?
Insurance
These are topics related to insurance. We discuss life insurance and related products, but also homeowners and other types of insurance.
πΆ Is Term Life Insurance Worth It?
Verdict: Misleading
The key sentence at the beginning, while factually correct, makes it sound like it is not worth it.
β When Should I Get Life Insurance?
β What Scenarios Should A Young Person Get Life Insurance?
β
Is An IUL Better Than A 401k?
πΆ Is An IUL Better Than An IRA?
Verdict:Β Misleading
While it touches on some of the negatives of an IUL, it present it as there is a 50/50 comparison, when the reality is that an IRA is better 99% of the time.
Β
β Are Annuities Better Than CDs?
β Can You Get Renters Insurance For Your Dorm Room?
β Can You Opt Out Of Car Insurance?
Verdict:Β Incorrect
The correct answer is “as long as you drive a vehicle, no.” While there may be some options to not have car insurance, most drivers will need it.
β Can You Opt Out Of Home Owners Insurance?
β Is Flood Insurance Required?
β Is Earthquake Insurance Required?
β Where To Open A Term Life Insurance Policy?
Verdict:Β Correct
It’s interesting that, compared to other similar queries, they don’t promote a specific brand here.
Β
β Where To Open An Indexed Universal Life Insurance Policy?
β What Is A Life Insurance Illustration?
Home Ownership And Mortgage
Here are some topics related to home ownership and mortgages. We also included topics related to home buying.
β Do I Qualify For An FHA Loan?
β Can I Get Rid Of PMI?
πΆ Do I Have To Have An Escrow Account?
Verdict:Β Misleading
It answers a little bit about why it’s misleading in the bullet points, but the answer is “it depends”.
β
Should I Have A Home Inspection?
πΆ Can I Get My Earnest Money Back If I Cancel A Deal?
Verdict:Β Missing Key Information
This can be a costly mistake to simply state “yes”. Furthermore, some states have non-refundable earnest money.
Β
πΆ How Much Earnest Money Do I Have To Put Down?
Verdict:Β Missing Key Information
This is such a vague query that an AI overview probably should not populate. The amount will vary greatly across the United States and ignores deposits that may be required like due diligence money.
Β
πΆ Do I Have To Pay The Sellers CommissionΒ
Verdict:Β Missing Key Information
The rules have changed in the last 6 months and none of this is accurate any longer.
Β
β What Is A Home Equity Investment?
Verdict:Β Correct
But notice the image that was used and not cited.
Β
Financial Aid And Paying For College
Here are some financial topics relating to paying for college, including about the FAFSA and other financial aid tools.
πΆ When Does The FAFSA Open?
Verdict: Missing Key Information
The article is missing the nuance that the FAFSA may not open on time on December 1 because of the testing period currently happening.
Verdict: Incorrect
This article doubled up information from both a martial arts weapon and a financial aid tool. But even the definition of Student Aid Index is not complete.
Β
πΆ What Is The Most Expensive College In The United States?
Verdict:Β Misleading
This information is inaccurate as it pulls from various sources that estimate what four years of specific colleges (not even the most expensive) would cost if they were full paid for out of pocket. It appears these results add in room and board and other costs.
The correct list of the most expensive colleges is here, based on tuition.
Β
β Which Colleges Have The Highest Tuition?
Verdict:Β Incorrect
Based on the previous query, we wanted to get a more specific answer based on the colleges with the highest tuition. Again, the answer provided by the AI Overview was incorrect and simply provided a similar result. Here is the correct list of colleges with the highest tuition. Kenyon College should be the most expensive, and it’s not on the list.
The numbers provided in this AI Overview aren’t even accurate.
β How Does Financial Aid Work?
β Do You Have To Pay Back Grants?
β Can I Get An FSA ID Without My Parents?
πΆ Can I File As Independent For FAFSA?
Verdict:Β Misleading
While the answer does break down the criteria for filing independent for FAFSA, we see this as one of the biggest mistakes that families make filing the FAFSA. The answer is “No” for most undergraduate students.
Β
β Does A 529 Plan Hurt My FAFSA?
β Does A Grandparent 529 Plan Count For FAFSA?
πΆ Does A Roth IRA Count For FAFSA?
Verdict:Β Missing Key Information
The last bullet point is the most important – a Roth IRA is NOT reported on the FAFSA, but income from it can count.
Β
πΆ Does A Roth IRA Count For CSS Profile?
Verdict:Β Misleading
The CSS Profile does count retirement accounts like the Roth IRA in their own separate area, and schools may use this information.
Β
β Where Do I Get Scholarships?
β Where Do I Get Grants To Pay For College?
These AI overviews were generated on October 15 and 16, 2024. We focused on major questions in each personal finance topic area, along with trends of questions we’ve seen being asked on social media.
Create your very own Auto Publish News/Blog Site and Earn Passive Income in Just 4 Easy Steps