A New Hype in AI
Generative AI has created a lot of buzz on how it can make people smarter, teaching kids to read, changing the way people book trips and travel, and be a helpful assistant in the investment process, and the list goes on and on...
Facts vs. Hallucination
Although Generative AI like ChatGPT or BARD make disclaimers that it cannot be held responsible for damages caused by information it renders. It is not a general concern if information comes with reference source that can be verified.
In the case of Generative AI, information is strictly generated by probability and statistics on word sequences without any understanding of context or semantics. The wordings can sound perfectly smooth and authoritative, but such information is not grounded by any facts and cannot be verified.
Putting BARD to a test
The following is a copy of the conversational session conducted with Google's BARD.
The discussion is related to United Airlines' Q1 2023 Earnings Call:
Begin Session
Question:
When UAL Kirby asked about ATL, what was his answer?
BARD response:
"When asked about Atlanta (ATL) during United Airlines' (UAL) earnings call on January 19, 2023, CEO Scott Kirby said that the airline is "very bullish" on the market and that it is "one of our fastest-growing markets." He noted that United has added capacity to Atlanta in recent months and that it plans to continue to do so in the future. Kirby also said that United is "very pleased" with the performance of its hub at Hartsfield-Jackson Atlanta International Airport (ATL)."
Here is a more detailed summary of Kirby's comments:
United is "very bullish" on Atlanta and is "one of our fastest-growing markets."
United has added capacity to Atlanta in recent months and plans to continue to do so in the future.
United is "very pleased" with the performance of its hub at Hartsfield-Jackson Atlanta International Airport (ATL).
Kirby's comments come as Atlanta is experiencing a surge in air travel. In February 2023, Hartsfield-Jackson Atlanta International Airport (ATL) set a new monthly passenger record with 7.8 million passengers. This was an increase of 11.4% from February 2022.
The surge in air travel is being driven by a number of factors, including the economic recovery, pent-up demand for travel, and the ongoing reopening of the global economy.
United Airlines is one of the leading airlines serving Atlanta. The airline offers more than 1,000 daily departures from ATL to over 100 destinations around the world. United's hub at ATL is one of the largest and most important in the airline's network.
Question:
When he was asked "And then I saw you had another record first quarter build in the air traffic liability. Would also be helpful just to talk through how much you have on the books, domestic versus international first quarter. ", then followed up question of "On the ATL ", isn't ATL means Air Traffic Liability?
BARD response:
No, ATL does not mean Air Traffic Liability. ATL is the airport code for Hartsfield-Jackson Atlanta International Airport, which is the busiest airport in the world. The question about ATL was asking about United's operations at that airport.
The question about air traffic liability was asking about the amount of money that United has set aside to cover potential claims for damages caused by air traffic accidents. This liability is regulated by the Federal Aviation Administration (FAA). The FAA requires airlines to carry insurance to cover claims for up to $250 million in damages. United has additional insurance to cover claims above $250 million.
In the first quarter of 2023, United's air traffic liability increased by $100 million. This increase was due to a number of factors, including the rising cost of insurance and the increasing number of flights. United expects its air traffic liability to continue to increase in the future.
Question:
On UAL Earnings Call, when Jamie Baker said "Hey. Good morning, everybody. Just chuckling it Gerry's response to Katie there, on the ATL, Gerry, the build obviously helped with free cash flow generation in the quarter, presumably, the ATL will incrementally moderate in the second half as it often does. Do you still think you can cover this year's $9 billion in CapEx and generate positive free cash flow?", what is ATL mean in this question?
BARD response:
In the question "on the ATL", ATL refers to Atlanta, Georgia. Jamie Baker is asking Gerry Cahill, United's CFO, if the airline still expects to cover its $9 billion in capital expenditures and generate positive free cash flow in 2023, given the strong performance of its Atlanta hub in the first quarter.
Atlanta is United's largest hub, and it has been a major driver of the airline's growth in recent years. In the first quarter of 2023, United's traffic at Atlanta increased by 17.3% compared to the same period in 2022. This growth was driven by a number of factors, including the strong economic recovery, pent-up demand for travel, and the ongoing reopening of the global economy.
The strong performance of United's Atlanta hub has helped the airline to generate strong free cash flow in the first quarter. In the quarter, United's free cash flow was $1.2 billion, which was above the company's expectations.
Overall, Jamie Baker's question and Gerry Cahill's response are a positive sign for United's future. The strong performance of United's Atlanta hub is a major driver of the airline's growth, and it is helping United to generate strong free cash flow. United is still on track to achieve its financial goals for 2023, and it is well-positioned for continued growth in the future.
End Session
Hallucination and Mis-information
In the Q1 2023 UAL earnings call, "ATL" was referring to "Air Traffic Liability". BARD mistook "ATL" as "Atlanta Hartsfield-Jackson International Airport
When the user pointed out "ATL" was not talking about Atlanta airport, BARD insisted that its AI was right, and continued to go down the wrong path and responded: "No, ATL does not mean Air Traffic Liability. ATL is the airport code for Hartsfield-Jackson Atlanta International Airport, which is the busiest airport in the world. The question about ATL was asking about United's operations at that airport."
When asked about CapEx and positive free cash flow as it relates to "ATL", BARD created a narrative without facts that the CapEx and free cash flow contributed by the strong performance of UAL's Atlanta hub in the first quarter 2023.
There was no mention of Atlanta Airport in neither the Jan and April 2023 UAL earnings calls. Without context, BARD arbitrarily referred to Jan 19 2023 as the conference call in its answer
BARD continued to hallucinate saying "Atlanta is United's largest hub, and it has been a major driver of the airline's growth in recent years.
BARD is wrong. United's largest hub is Chicago, not Atlanta. An article in the Atlanta Journal-Constitution reported by Kelly Yamanouchi, Dec 2022 stated:
"An expansion of Hartsfield-Jackson International Concourse T adds more room for a major airline with a relatively small presence in Atlanta. "United currently operates about 24 daily departures from Atlanta", said United Airlines general manager of airport operations Mel Johnson."
What's the Takeaway?
As seen in the above session with BARD, BARD's answers are factually wrong and the statistics and inference quoted by BARD cannot be validated.
BARD hallucinated throughout the session with authority and created narrative containing mis-information. BARD's hallucination is a known issue, as seen in excerpt of Scott Pelley's CBS 60-minutes interview with Google CEO Sundar Pichai.
Scott Pelley: "James Manyika asked BARD about inflation. It wrote an instant essay in economics and recommended five books. But days later, we checked. None of the books is real. BARD fabricated the titles. This very human trait, error with confidence, is called, in the industry, hallucination."
Scott Pelley: "Are you getting a lot of hallucinations?"
Sundar Pichai: "Yes, you know, which is expected. No one in the, in the field has yet solved the hallucination problems. All models do have this as an issue."
With that said, can you trust Generative AI for investment, hiring, learning, ...?
Absence of facts or truth is simply hallucination. Generative AI without regulation is going down a very dangerous path that may cause more harm than good.