Jahanara Nissar

NVDA: Has the H100 user wave crested?

|

NVDA bulls appear to have decided to sit out the immediate aftermath of the earnings call and prepare for a temporary pullback. They appear secure in their belief that the H-series continues to be supply -constrained, and with the added insurance that the beefier B100 is just around the corner. So why worry? Let’s buy the dip, say the bulls. But what if the tip of the spear in mass-market AI development is no longer chasing beefier GPUs and ever larger GPT-type foundational models?

For mass market AI app developers, the divergence between H100-GPT4 pricing and a path to profitability is too daunting, we believe. And so, they necessarily must walk from H100 for now and seek cheaper solutions. If we are right, then the supply-demand imbalance the Street infers from upstream checks could be closer to an end. And that will not help the bull thesis.

Mass-market AI applications, such as tech support chatbots and email summaries, need to seek out an alternative path. We believe there is intense development activity ongoing to fine-tune GPT4 class of LLMs in order to fit the vector database into NVDA’s older gen A100, which happily enough seems to be in surplus. Our checks show A100 servers running open source LLMs are less than a tenth of cost of GPT4-H100 in terms on $/token. And that is unbearably attractive to AI app developers. We believe the initial wave of H100 users has crested.

The print/guide provided today may not be quite as relevant as qualitative commentary on out-quarters. NVDA management is likely to provide the Street with just enough juice to model next year up y/y. Purchase commitment for Fy26 is likely to go up vs. the ~$1bn commitment disclosed a quarter ago. The Street appears to be modeling NVDA’s Fy26 Data Center revenue growth anywhere from up 20% to up 50% based on supply-chain checks. We think upstream supply-chain signals are not a reliable indicator of future growth when there are shifting trends downstream with AI developers.

We do not think NVDA is trading on fundamentals as expectations for next year seem unmoored from business realities downstream. We take our previous PT of $425 (link) from 3 months ago and index it up by the Sox up 17% to get to a new PT level of $500. We will need to see NVDA stock give up all its ytd gains before we get interested on the long side especially as, on a macro level, a whiff of inflation is making long-duration secular names such as NVDA incrementally less desirable to investors.

–xx–

Development costs are simply too crushing to all but a handful of hyperscale AI-service providers, in our view. Even at hyperscale AI-service providers, we doubt if their services/applications are close to profitability. Certainly not at Google GCP – external AI users have essentially been getting free access to Bard/Gemini. Microsoft’s Copilot priced at $30/month may not margin accretive, our industry checks show. But these giants have the financial muscle to take losses while they seed a new market. Smaller 3rd party app developers do not have that luxury. So how are they going about it?

The more innovative app developers appear to be pivoting away from supply-constrained AI-hardware. The tip of the spear appears to be pointing away from expensive H100 servers and towards older generation A100, for which we think there is surplus supply. We think there is now an active secondary market for A100 servers. This has encouraged a new crop of relatively unknown boutique CSP to enter the AI-fray and act as price-spoilers to the CSP incumbents

We believe rental pricing of A100 servers has been dropping rapidly and is now a fraction of the H100 servers. This acts as a powerful motivator for app developers, who were previously dabbling with H100 but with limited success due to high pricing, to gravitate to the older generation A100. A race to the bottom in pricing – this has been ethos of tech innovations over the past 4-5 decades. This time is no different.

Who is the incremental buyer for the B100? When the beefier B100 hardware becomes available in a few months, if the H100 user base is already cresting, who will be the incremental commercial buyer for hardware that is more expensive than the H100? We find it hard to believe hyperscale CSPs could acquire the B-series with the same gusto they did the H-series servers.

There must be a reason Jensen has been, of late, cozying up to national leaders. Maybe it takes the backing of a national budget to fund the enormous outlays B100-based data centers may require. We note in passing that the estimated combined capex of the top 4 US-based CSPs sits just below the #2 national defense budget (China), and larger than the combined budget of the #3 and #4 nations (Russia, India). At some point of time, commercial CSPs need to disclose profit metrics; they have been silent on that front so far. Loading up on more capex does not help. We think Cy25 will be a year of capex digestion as the top 4 US players allow costs to run-off via depreciation.

A new phase: We believe the initial spurt of activity in Gen AI is fast maturing and is entering a new phase. The period of intense experimentation at all costs may be behind us. Our checks across the industry show mass-market application developers are moving away from experimentation to a new phase of discovering paths to profitability. This new phase, while just as exciting and innovative, may not be quite as fast-moving. And the path to profitability, may not run through NVDA’s most advanced GPUs

At the very high end of the spectrum of applications are the ones which hold the promise of dramatic productivity gains in the near term. These applications, such as GitHub CoPilot, could well be profitable on the existing high-end solution, i.e. GPT4 running on NVDA’s H100.

The H100 user wave may already have crested: However, proliferation of Gen AI applications beyond just the most lucrative end, we think necessarily requires innovation in lower cost hardware and open source LLMs. We think developers in mass market applications are moving away from NVDA and OpenAI’s cutting-edge solutions – they simply do not see a path to profitability. In our conversations across private developers, we think the wave of GPT4-H100 users may have crested. We think 3rd party app developers have been moving away from the GPT4-H100 combination in the past few months.

Signs of cresting user base: 1) OpenAI is having to make drastic cuts to pricing every three months, 2) VC-funded startups on Y-Combinator platform (Altman’s alma mater), we hear are being encouraged to use OpenAI’s GPT-models instead of cheaper open source models, 3) Microsoft CoPilot, our checks show, is handing out free seats to app developers, 4) Google GCP AI appears to be in no hurry to move away from free access to external users.

In search of cheaper hardware: So where are mass-market AI app developers headed? To AMD? To proprietary solutions from hyperscale CSPs? No. We don’t think so. The non-NVDA solutions are far from the plug-and-play stage. These developers have no choice but to continue working within the NVDA family of products, for now. However, we think they may have discovered that a potential path to profitability runs through NVDA’s older gen product, the A100. And why is that?

A secondary market has emerged in A100: We think the A100/80GB is already trading on the secondary market, and with it, a drop in hardware cost and server rental pricing. Less than three years since launch, we think there is surplus of A100 in the market. Hourly rates for A100 servers at data centers have come down over the past year. A100 is offered at a fraction to H100’s hourly rates. Our checks show A100 servers running open source LLMs are less than a tenth of cost of GPT4-H100 in terms on $/token. And that is unbearably attractive to AI app developers. And so, many developers are taking their H100 models and are trying to optimize them to fit on A100 servers.

Price spoilers enter the AI DC market: The clearest signal of surplus A100 in the recent emergence of price-spoilers entering the AI data center market. Boutique DCs with sub-$billon annual revenue and annual capex of only ~$100mn have begun to enter the AI data center market with the explicit goal of poaching users from incumbents. Some of the outfits are highly profitable and prefer to stay that way after entering the AI market. We think they are managing to procure A100 servers at super low prices, thus ensuring continued profitability as they scale up their AI customer base. This is very good news for 3rd party AI app developers.

The H100 too will eventually go into the secondary market perhaps sooner than the A100. As supply of H100 catches up with demand and as the initial wave of H100 users crest, we think the H100 too is likely to go into surplus supply. While it took ~2years for the A100 to go into surplus supply, we think the H100 may get there sooner due to the steeper increase upstream in H100 supply capacity vs. the A100.

Net/Net:

  • While H100 may seem supply-constrained to the Street going purely by upstream supply chain checks, our downstream checks seem to indicate that the initial wave of H100 users may have crested as the pricing is too rich for most mass-market Gen AI applications
  • Mass-market AI app developers need to 1) find innovative ways to take their GPT4-H100 models and adapt them onto A100 servers running LLMs smaller than GPT4, or 2) suspend all development until H100 supply goes into surplus.
  • We expect H100 to go into surplus once hyperscale players such as Meta and Microsoft Azure run into excess capacity and start dumping servers into the secondary market. We think Microsoft Azure could be close to hitting excess capacity. Why else would they be signing up new users of Copilot for free?

We do not think NVDA is trading on fundamentals as expectations for DC growth next year seem unmoored from business realities downstream. We take our previous PT of $425 set 3 months ago (link) and index it up by Sox advance up 17% to get to a new PT level of $500. In other words, we will need to see NVDA stock give up all its ytd gains before we get interested on the long side.

AMD, SOX: Fed impact could magnify downside to AI names

|

Movement in Treasury rates following FOMC meetings typically has material impact on tech stocks. The FOMC event today we think could be less about the timing of Treasury rate cuts and more about an acknowledgement by the Fed that the underlying economy is running stronger than expected, and that the odds of a recession have come down. In our view the market reaction to such a take-away would be for investors to shift allocation towards economically sensitive cyclical names, such as industrial and commodity names, and away from high-growth secular names, such as the AI-fueled semis and software names.

The AI-exposed names which reported yesterday, AMD and GOOG but perhaps not MSFT, may be just a tad off in the delivery of good news to investors. This slight roll-back in fundamentals could be compounded by the Fed event today and could result in trimming of recent gains by AI-exposed names.

The significant runup in these names since early Nov, following the Fed’s November meeting has been in no small part due to a significant drop in real rates since Nov (Exhibit 1), in our view. Falling real rates provide tailwinds to equity multiples of secular growth stocks. Could there be a further drop in real rates from current levels? We doubt it. If anything, real rates could move up if the Fed were to lower the odds of a recession. Rising real rates could provide headwinds to multiples, just as they did in the Aug-Oct period of rising real rates. Even if real rates move sideways, as they have been doing of late, the loss of downward momentum in rates could provide loss of upward momentum to equity multiples.

We would look to trim positions in high-growth AI-exposed names and add to cyclical names in the memory/storage and Industrials/Consumer sectors. We would trim position in AMD and look for a 10%-15% pullback from yesterday’s close. SWKS would be a good cyclical name to get behind.

Thoughts on AMD: Into a slightly disappointing AMD report, the stock could have held up well into investor anticipation of upside to MI300X outlook provided yesterday. However, we think the Fed’s stance today may compound the modest earnings disappointment. We would be wary of adding to position at current levels; we would rather wait for a 10%-15% pullback.

We note that excluding the MI300X product, AMD’s 2024 annual outlook implied by management’s qualitative comments points to revenue flat to down slight for the

year, not exactly a ringing endorsement. The company appears to be muddling along entirely on the strength of the MI300X product line.

Macro discussion into Fed event today: There has been a lot of discussion on the Street with regards to the Fed being under pressure to cut nominal rates just so real rates do not spike to the upside in response to falling inflation. We think this is something of a spurious argument. We think the Fed has the luxury of stringing the Street along without having to provide timing of cuts. However, it needs to give a reason as to why they are in no hurry.

We think the critical signal coming out of this meeting is not the timing of rate cuts, but rather an acknowledgement from the Fed that, the surprising strength in the economy, despite high interest rates, reduces the odds of a recession in the medium term. Hourly wages growth, which had been declining most of last year, seem to have stabilized in recent months (Exhibit 2). Consumer confidence metrics show unusual strength. If so, where is the urgency to cut rates?

Real rates have already declined, rather dramatically, in response to dovish Fed Chair comments following the FOMC meetings last November and December. On a 2s, 5s and 10-year basis, real rates have already dialed in rate cuts at some point of time in the future; they may be relatively immune to the exact timing of the cuts.

If anything, there is a case to be made that the Fed may hint at downside to the number of cuts penciled into the SEP at the Dec FOMC meeting. The 1st estimate of Q4 GDP has come in significantly ahead of market expectations prevalent at the time of the Fed’s December meeting. Q1 GDP estimate out of the Atlanta Fed too is running hot, close to the Q4 level. In other words, the economy shows no sign of cooling down. So why would the Fed be anxious to cut rates?

Just as the Fed was loathe to raise rates in the absence of data in the days of high inflation (2022-23), one would think the Fed would be just as loathe to cut rates in the absence of data signaling signs of cooling economy and deteriorating labor markets, neither of which exist today.

That the Fed could indicate rate cuts in anticipation of rising real rates (as inflation cools further) is just not a good enough reason, it seems to us. This argument is especially ill-conceived as real rates have already fallen to a long-running median level (dotted line in Exhibit 1). If on the other hand real rates were running close to where they were back in Oct (prior to the Nov FOMC meeting), we would imagine the Fed might have been a bit more nervous allowing real rates to gallop further. This is not the case today.

If there is something to worry about, however slightly, it would be the nonfarm payroll on a 3-month averaged basis (Exhibit 3). This data has been trending down, as it should. In recent months though, the 3-month moving average has cut below the pre-pandemic average monthly level (dotted line in Exhibit 3). And yet the level (~165k) is still quite a bit away from getting into the negative territory, which would then need a response from the Fed. But the downward trend bears watching as it creeps closer to zero. So, the Fed will wait for more data from the labor markets before making a move.

Net/Net: We think the Fed is likely to acknowledge the surprising strength in the economy even as inflation metrics trend closer to the Fed’s target. In such an environment, standing pat and letting fed rates stay where they are, may be the best strategy.

If the downward movement in real rates is behind us (i.e. no further gains in the multiples of high-growth names) and if the odds of a recession have come down, then it seems to us that a good portfolio strategy would be to trim high-growth AI names into their recent outperformance and then allocate capital to cyclical names, which have underperformed the indexes over the past three months.

image
image
image

AAPL: Looking for upside to iPhones and a peek into AI

|

Apple goes into its earnings call today with the weakest investor sentiment in recent years. Four consecutive quarters of negative growth in FY23 and the 1st quarter in FY24 guided flat have taken their toll on investor sentiment. We were in the minority calling for negative revenue growth in Fy23 as early as Oct’22 (link). Going into the 4Q23 earnings call, the stock hit our previous PT of $170. After calling for downside to investor expectations all through Fy23, we turned the corner in mid-Dec’23 into what we felt was excessive investor pessimism (link), and raised our PT to $220.

While an overall slowdown in iPhone sales weighs on sentiment, investor expectation of a challenge from Huawei’s Mate60 tipped investors into outright depression. We took a contrarian view. We felt investors were being overly pessimistic in their China assessment. Ex-China too, we felt iPhone had the opportunity to gain share from Samsung’s aging Galaxy S23. In recent weeks, reports from market research firms have largely confirmed our assessment of the Dec quarter. Rather than lose share, iPhones seems to have gained share in China and also worldwide as Huawei and Samsung failed to live up to expectations.

As for the March quarter and for the full year, investor sentiment seems just as low on continued China fears and unconfirmed reports of component cuts. However, our checks show that TSM raised 3nm node utilization to full capacity. As there are no major 3nm customers outside of Apple, we think Apple may have raised 3nm wafer starts at TSM, potentially a positive for iPhones and Macs. 

Fears related to iPhone weakness need to be addressed by Apple management. We believe they will. But that alone will not do it. Apple needs to demonstrate itself as a full participant in the AI revolution, not just in applications as many Apple bulls are hoping for, but in Apple silicon for running full-fledged LLM workloads on smartphone and PCs. The M2 and M3 chips designed for Macs show tantalizing promise. It is unclear to us as to what is management’s strategy for AI. But the potential for a significant new growth vector exists.

We model FQ1 at $120bn/$2.16 vs. consensus at $118bn/$2.11. We model FY24 at $391bn/$6.4, revenue up 2% vs. consensus at $395bn/$6.62, revenue up 3%. We are long into print.

iPhones in FQ1: In our Dec note, we raised our iPhone expectations to up 7% y/y vs. consensus up 4% on higher-than-expected strength in China and worldwide.

Going by reports from market research firms, iPhone shipments in CQ4 may have worked out largely inline with our thinking – no share loss in China and worldwide. According to market research firms, IPhones gained share in China, despite the Huawei challenge. Other China brands seem to have lot share to Huawei. This implies that Huawei, while still an important player in China, is largely limited to mid-range smartphone models. Worldwide too, a similar story seems to have played out, for the quarter and for the full CY. iPhones gained share from Samsung to become #1 worldwide by unit volume.

How did iPhones manage to get on top of the competition? 1) On the strength of its balance sheet, we believe Apple has been able to out-finance its competition by offering easy terms to telcos and distributors. Samsung’s loss of share we believe has been due to constrained finances and their inability to help telcos stock up shelves. 2) As for Huawei, we believe poor yields at SMIC 7nm process constrained the supply of the Mate60 and prevented Huawei from following through with the initial success of its launch in early Oct’23. The challenge from Mate60 to iPhones may have been transitory

Apple may have a silicon solution for AI: Whereas Apple bulls have been sending up trial balloons in the form of hopes for Gen AI applications in next gen iPhones, we think Apple’s position in AI could be far more fundamental.

After a decade of integrating its operating system with its cpu, gpu, neural-network and memory, we think Apple may well have the single best silicon capable of performing inference workloads at the lowest power envelop. If not all inference workloads, at the very least transformer functions.

The AI investor base mostly avoids considering the huge cost of running today’s mainstream AI silicon on liquid-cooled GPUs. We believe there is a role for today’s large-cluster liquid-cooled GPU servers in specialized functions - LLM training and large-scale inference workloads. However, for mass market inference workloads and for training SLMs, we think the AI marketplace needs low-cost, air-cooled traditional CPUs. We think the yet-to-launch M3 Ultra at 3nm or the M-series follow-on at 2nm may be that long sought-after chip. And if so, Apple stock has a long way to go.

We called out this possibility going into the M3 launch (link) back in Oct’23. While Apple has avoided referring to its AI plans at the launch event, company management has mentioned in passing, the silicon’s potential for transformer workloads. The M3 Pro supports 126GB of ‘unified memory’ (the gpu shares address space with the CPU) vs. NVDA H100’s 80GB of ‘discrete’ memory (which is more power hungry than unified architecture). The M3 Ultra, yet to be launched, likely features 2x126GB (vs. AMD’s MI300X at 192GB), spread across two chips, thus providing massive memory for loading LLMs.

And it is not just in the hardware specifications. Apple silicon is fully integrated with Apple’s software stack. Also, the open-source community ported PyTorch onto

MacOS. Whereas Intel and AMD talk about ‘AI PC’, Apple’s M2 Mac Pro has already been in use as an AI PC for over a year. Meanwhile, Microsoft Windows OS is yet to provide hooks into x86 PC solutions due to lack of Windows’ integration with Intel’s OpenVino or AMD’s ROCM.

Net/Net:

  • For the stock to shed investor pessimism, Apple management needs to demonstrate strength in its core iPhone business. We think it can.
  • While very much out of character for Apple’s management style, Apple needs to give investors an advance peek into its AI plans, especially as it has under its belt, Apple silicon which has the potential to beat the AI incumbents in mass market AI workloads.
  • We are long into print.

ARM: The hidden hand of AAPL

|

Anxious to shed its image of being overly dependent on the mature smartphone market, the narrative from ARM management at the earnings call last night leaned heavily towards cloud server CPUs. However, we think the strength in royalty revenue in the December quarter was driven by the more mundane ‘annual refresh cycle’ of premium smartphones as smartphones ‘returned to strong growth in Q3’.

We believe the y/y increase in royalty revenue is based largely on Apple’s return to growth, a prospect few on the Street are willing to embrace. Into an environment of overarching negative investor sentiment, we turned the corner on AAPL in December (link) as we raised our PT to $220. We reiterated our constructive view into Apple’s earnings event (link). Strength in ARM royalty revenue gives us more confidence not only in iPhones but also in a possible turn-around in iPad.

We think the strength in royalty revenue was anticipated by ARM management. On the other hand, the upside surprise to print/guide, in our view, came from licensing revenue from the cloud server CPU market – specifically, from NVDA’s GH200. However, we do not expect NVDA’s licensing agreement with ARM to translate into material revenue for NVDA’s ARM server product, in the medium term, as Microsoft is yet to complete porting its Windows Server OS onto NVDA’s cpu.

ARM’s royalty revenue growth driven by Apple’s emergence from the doldrums:

While management talked up share gains in server CPU, the role of smartphones, contributing 35% to ARM’s royalty business, cannot be de-emphasized.

ARM’s FQ3 royalty revenue was up 11% y/y, even though overall chip units shipped was down 3% y/y. While not calling it out by name, ARM’s royalty growth in FQ3 had a lot to do with Apple iPhone’s return to y/y growth, in our view.

ARM’s FQ4 guidance of royalties up 30% y/y is also strongly influenced by Apple, in our view. That may come as a surprise to many given Apple’s soft guidance for its March quarter.

We believe Apple started a strong ramp of ARM-based M3 wafers in January at TSM’s 3nm node in preparation for the launch of next generation iPads likely in the June quarter. We think TSM’s strong January revenue reflects strength at its 3nm node and gives credence to our iPad thesis. ARM’s royalty growth in March quarter and beyond, we think in part is due to Apple’s iPad refresh and the higher royalties of ARMv9 at 3nm vs. the previous generation of iPads at 5nm.

The surprise upside to print/guide driven by NVDA: The upside to ARM’s print/guide is due to unexpected strength in its licensing business, in our view. ARM cfo spoke of upside to print/guide from selling ‘additional licenses’ to AI end-market that was ‘just not in our plan and not anticipated’. We think the upside surprise in Q3 print and to Q4 guide comes from the licensing revenue from NVDA’s GH200 superchip. We believe the GH200 is targeted for shipment to Microsoft.

According to our checks, NVDA is licensing ARM in advance of actual shipment of GH200 servers. The timing of the shipment will depend on when Microsoft is done porting Windows Server OS onto NVDA’s ARM platform. We believe Microsoft is not ready with the solution. The timing of the shipment of GH200 to Microsoft is not clear.

Net/Net:

  • ARM management appears confident of its royalty revenue growth. We expect to see ARM’s royalty strength translate into revenue growth at Apple.
  • ARM management appears circumspect in its licensing business outlook beyond the current quarter. We do not expect upside surprise in ARM’s licensing business to translate into strength at NVDA’s ARM server business in the medium term.

Latest Posts

Categories

Calendar

April 2024
M T W T F S S
1234567
891011121314
15161718192021
22232425262728
2930