Open-weight vs open-source, new opportunities, and 2024 in review

Ellianna

It's been a huge week so diving right in, what do you think of DeepSeek?

Michael Frazis

DeepSeek came out over a week ago now and has already had a pretty big impact.

Last Monday, Nvidia had its biggest one-day loss of any stock in history.

The headline was that DeepSeek spent $6 million USD to train their model, which is a fraction of the Capex plans of other companies.

To add some context, Meta just reported plans to spend $60 to $65 billion USD this year, for example.

You're talking like four orders of magnitude difference. It was a bit cheeky for DeepSeek to highlight it because they're not including their hundreds of staff, that they apparently pay the highest salaries in China to, and they certainly did have a lot of Nvidia chips.

This was originally a hedge fund that was known for having large Nvidia clusters.

So that headline number is misleading, but it did surprise everybody because this was a model that was certainly done a lot cheaper. And it's also available for free.

It's really put into question the business models of companies that are building foundational models and planning to monetize.

If you can create a state-of-the-art model for a fraction of the cost, then why are the big tech companies spending $50 billion USD+ each on capex a year?

It really raises questions around that.

Ellianna

For those who don't know, what's the difference between open weight and open source?

Michael Frazis

Open source has been around almost as long as software. In open-source, people put their software online, allow others to use it, improve on it, commit code, which would then be approved by the community, or realistically community leaders, which would then improve the product for everybody.

Much of the world is built on open source, notably the Linux community and most servers.

But this is not an open-source model. We don't know the code they used or exactly how they did it. They have released some papers and they have given some information on it.

Instead, it’s open weight.

And that's interesting because it means that it's free, it’s in the public domain.

You might not know exactly how it was trained, but anybody can host it.

If you have a big enough, powerful enough desktop, you can download the model, certainly smaller versions use them, for free. Or you can host it on AWS or Microsoft Azure.

There was a bit of controversy around how it was trained.

It seems they were using ‘distillation’ with OpenAI's model to train their own, check results, basically create new synthetic data to improve their own model.

So advanced models can be used to improve other models.

And if it's open-weight, everybody else can use that model to increase the power of their own. So once an advance like this is in the public domain everyone benefits.

OpenAI didn't seem particularly happy about that, but of course, OpenAI scraped the whole internet without asking anyone’s permission, as well as every published book.

Now it’s in the public domain, DeepSeek is an asset for the rest of humanity forever. This level of language model will now be free. And we're not going back.

Ellianna

What do these terms mean, distillation and mixture of experts?

Michael Frazis

Mixture of experts means you train one model to say answer math problems, another one for coding, another language tasks, then use a language model to figure out which one’s best for a particular query.

 And that's how you can get really good performance in those domains.

It also means even though the model itself was about 670 billion parameters, at any one time probably only 37 billion were being used. And it's fast. OpenAI uses this.

On distillation, we don't know exactly what's happened.

OpenAI is confident they have evidence DeepSeek was using their API to train their own. And presumably those requests were all logged, so they should know.

It again raises questions about the defensibility of some of these business models.

To step back, there’s a big difference in the way that people are monetising large language models.

Companies like Meta are going down a more open route. They're not directly charging for large language models, but they are benefitting from the community that surrounds it.

Their main game is improving their ad spend and generating revenue and profits that way.

When it comes to winners and losers, the losers will be anybody that's planning to charge a subscription for a language model, which is now effectively free.

At best maybe you can expect the state-of-the-art open weight models to be six months behind the absolute state of the art.

There are some pluses, but you there's a long tail of language models. Like where does this leave Mistral in the EU? Where does this leave Anthropic?

What's the value of being a fourth, fifth, sixth, seventh best model? And there's a lot more than seven companies building foundational models.

And I guess what spooked the market was, what does this mean for capex and chip spend?

All these companies have been buying the latest Nvidia chips and spinning up their own clusters. There's been a massive imbalance between demand and supply, demand has massively outstripped supply for the last two years.

Now what happens next?

With such a good open-weight model in the public domain, how can a mid-tier provider justify spending money on capex? And these companies are all loss-making.

They're all reliant on future rounds of funding, and that funding could dry up. Users will consolidate around the very small number of winners. And there’s a much smaller number of winners now that DeepSeek has come out.

There's a good chance those companies can't raise the money they need to stay competitive in the game.

There are also clear winners.

Even though these open-weight models are free, they still to be hosted somewhere.

You still need to put them on AWS, Microsoft Azure, Google's cloud platform or one of the smaller companies. This could potentially drive demand for those services.

There's another set of winners, which I think the market is not appreciating.

For the more traditional software companies, a cheap model opens a lot more use cases. There's a good chance in the next year or so those use cases start to take off.

Ellianna

Could you explain the point about unappreciated software companies?

Michael Frazis

Basically, all software companies are restricted by the number of people on the planet.

If you're Salesforce, in the best possible outcome, your maximum possible market is everybody on the planet, that every single person has a Salesforce account.

Obviously, that's unrealistic, there's only a certain number of business workers, only a certain number of those will have use for a CRM. And of those Salesforce is only one competitor.

But the market size has always been tethered to population. It's always been tied to the number of people with a computer sitting down and doing work.

If you can make an AI sales agent, then that is unrestricted.

You could spin up 20 of these overnight for the marginal cost of computing power, which is a lot cheaper than hiring somebody. You’re untethered to the population.

 And that's really the bull case.

It's not just Salesforce, which is not even the best example.

Each one of these agents or operators could be a security endpoint, each one of will have to be protected. Each of these agents is going to generate a ton of data and will be interacting with other agents. In many instances that data might have been previously restricted to the amount that two people can generate.

Now with AIs with potentially human level intelligence, population is no longer a cap to those original market-sizing assumptions.

All that data has to sit in a database somewhere. That database is going to be charged for and is going to need monitoring. This is bullish for monitoring software.

All the standard software subscriptions that a knowledge worker would use, these agents may need as well. So, the software opportunity is way bigger than what people are thinking, though people are starting to connect the dots. You could see that in the market reaction.

When Nvidia was down 17%, software was largely unscathed and bounced hard the next day.

So I think the market’s slowly starting to figure out the opportunity here.

The race for a good foundational model is getting somewhat mature.

There'll continue to be advances, but it's clear that there's going to be a handful of companies doing it. The middle and lower tier are probably not going to make it. And there will be very amazing, cutting edge, open weight models available for free.

But this other part of the demand curve is really in the very first innings.

It’s something we're looking at as we’re trying to figure out what will perform well in 2025.

Of course, it’s just a possible outcome, so the jury is still out on how strong the thesis is, and the best way to play it.  

And any price moves will have to be justified by the numbers and fundamentals that actually come through. But it's a really interesting piece of what's going on right now.

Ellianna

Speaking of spending in the AI space, I wanted to talk about Stargate.

Michael Frazis

It's kind of interesting.

It was released a day after DeepSeek and maybe marks the end of the first phase of this AI boom.

In the last two years, every major tech company has been proudly emphasizing how much they're going to spend. When they announce big spending plans, and their stocks went up.

That's quite unusual, if a car company announced a massive capex expansion, there would be huge questions around it.

Meta released results last night and they're still sticking by $60 - $65 billion USD cap expand. That's way ahead of what, say, an automaker would spend.

I do wonder if it marks the last moment of that leg of the cycle, particularly because it was Masayoshi Son.

Ellianna

And who is Masayoshi Son, for people who may not know?

Michael Frazis

He's one of the most interesting investors because he's made more money than almost anybody.

But he makes the dumbest decisions.

He's some mix of the best and the worst investor in one person.

If you invest in a random company that he backs, you're probably going to lose all your money.

Ellianna

Then what's made him successful?

Michael Frazis

Well, he invested in Alibaba.

I think it was $20 million. And then just held onto every single share and became insanely wealthy. He runs SoftBank, which is a leading Japanese telecommunications company.

He's done a small number of investments that worked and held onto every share. And a very, very small number of those have been some of the best investments ever made.

He does have his habit of coming in at the top of market cycles.

He did it in the dotcom crash as well. Nearly got completely wiped but held onto enough equity to ride through. He did it in 2020, 2021. He was one of the leading players that were pushing up valuations then and had a terrible impact on the market.

He was funding enormous checks into companies with no moat, many of which no longer exist anymore.

He comes in as the most aggressive guy.

But it's kind of interesting because when you think about the investments that worked, I wouldn't say that was his strategy with Alibaba. That was a much smaller company with a huge opportunity. Now he's coming in relatively late into a capex cycle, it’s not the same.

There was perhaps a moment two years ago when it became clear chips were going to be so valuable and so vital to the whole tech ecosystem.

That was the moment. That was the moment where it was like, well, we're right at the beginning.

There was nowhere near enough capacity to build the chips that everybody wants. And that was when companies like Nvidia went up more than 10 times, justified by earnings and profits. It's not actually that expensive on this year's earnings. That was the time.

Now it's different, the big tech companies are probably not going to grow their capex in 26. They've done these massive capex plans, and that growth is possibly over, even though they will still be enormous annual numbers.

So, Masayoshi Son's there, waving a $500 billion number in the air.

The odds of that spend actually happening are quite low, and it's a much smaller actual commitment.

An open-weight model is a real challenge to that model. I mean, I've already started using it personally. Another advantage of the open weight model that I didn't mention was that you don't have to host it in China. You don't have to send your data over there.

You can put it on your own hardware if you want. You could do it as a side project and then nobody could ever take it away from you. You don't have to connect to the internet. It's just going to give you whatever answers you need.

I saw someone saying it'll cost about $6,000 USD to get your own system that will run the full model. We could do that in the office.

You don’t need $500 billion.

Ellianna

Do you think that that gives DeepSeek an advantage beyond being free and open weight over other AI available on the market right now?

Are people going to want to use it more instead?

Michael Frazis

Absolutely, though perhaps hosted in the West. And there will be a new generation very soon.

The Chinese obviously have their own biases built into the model. But maybe that’s worse with OpenAI.

There's a huge amount of political influence over OpenAI’s results.

It’s not just US politics which are affected, with models more critical about Republican candidates and much kinder to Democrat candidates.

But it also affects user queries on things like history, for example when Google’s image generator changed the skin colour of historical figures in images and things like that. It makes the models dumber, reduces their understanding of the world.

There’s this pervasive political influence.

These open-weight models don't necessarily have that. And it's yours to use. So, it is a huge advantage to open-weight in addition to being free and being able to host it wherever you want.

It really is a threat to those companies.

I think all the cool kids in the space will be using the open-weight models.

Maybe some people use OpenAI because potentially it's more reliable or enterprise grade. But the cool kids are going to want to use open-weight.

Ellianna

ASML reported their earnings earlier this week and they've doubled their bookings.

Their guided earnings for the next year are projected at 46%, let's talk about that.

Michael Frazis

That's a strong number.

Again, we're relatively late, certainly a couple of years into this capex cycle and they're still posting really high numbers.

ASML is pretty far down the purchasing chain. Think of customers like Meta, Microsoft, and Twitter, they're the ones putting in purchase orders for Nvidia. Nvidia is then purchasing capacity at Taiwan Semiconductor. And Taiwan Semiconductor is then deciding how many lithography machines to buy from ASML.

That's the chain.

So any slowdown will take a while to filter through.

If the cycle turns, what will happen is a few companies will stop getting funded.

That already started to happen last year.

They’ll then dump their excess Nvidia chips on the market. You'll see the second-hand price of those chips fall. You'll see the cost to rent those chips fall in real-time.

This has all happened multiple times in the last five years. The cycles have been really short and fast.

Then that will affect Taiwan semiconductor. They'll make less chips for Nvidia, and perhaps scale back orders from ASML.

ASML is going to be a very lagging indicator in this, but they’re the only company that can do what they do in the West.

I've written a few times that Biden's decision to restrict Chinese access to leading chips was a huge mistake.

The reason is that was the Chinese are going to do it anyway.

And it's better that they buy Western chips because then the capex, revenues, profits and increase in capital happens on the balance sheet of Western companies, which can then maintain their lead.

If there was some kind of hostile conflict, you're much better off having big, strong Western companies dominating that industry.

Ellianna

Let’s change topic, so the fund ended the year up 56.5%.

Walk me through how 2024 went.

Michael Frazis

Yeah, the two halves were completely different at almost exactly the six-month mark.

The first half of the year everything seemed to go right.

Aussie biotech's were ripping with companies like Clarity going up several times. We had a number of our classic explosive growth, true customer love stocks. They all pushed to new highs. Then we had semiconductors as our core position as well. All of those companies did well.

Around June, there was a change in regime.

Semiconductors kind of topped out and, other than Nvidia and Taiwan semiconductor, the vast bulk of them sold off significantly.

There was also a huge number of reversals in those explosive growth, true customer love companies.

Whether it was in Australia with Droneshield, or Celsius, Elf, Transmedics in the United States.

There were a number of companies that dropped 50%, 60%, 70%.

We were lucky because we've been using these AI models to build risk systems that basically flagged all that stuff early and allowed us to exit those positions, often with substantial profits, before they imploded.

We still ended the year in the top five in Australian equity fund managers, despite so many of our stocks having major reversals.

So that was positive.

But it did feel like we were playing defence for the last six months, which is a lot less fun than when everything is going up.

But it was a real proof of concept because we monetized some huge gains, like 150% plus gains in companies like Droneshield and Transmedics that then ran into serious market trouble.

If there's one thing we learned over the last five years is that the downside in these high performing growth stocks is not 30%, 40%, 50%, like it might be in a bank or an industrial company. When they reverse, they reverse really far and fast.

The market moves a lot faster these days. There are whole cycles in a year.

I do have a feeling that like things like Twitter, the easy access of information, Reddit, of options trading; all that kind of stuff really magnifies it.

When a stock captures the imagination of the market, it might double, triple, quadruple. And then the next 10 years of success becomes baked into the price.

And then as that starts falling, all those price-based and momentum-based investors quickly drop off. The options expire and the hedgers sell the stock.

The way we're approaching this now is that we really want to harvest those gains from these market rips. We’ll still be long-term investors in companies, just only at certain times.

We went through every stock that we owned and realized that if we had sold things that went up 4x, we would have dramatically outperformed.

If we sold things that went up 2.5x, we would have done less well, but still dramatically beat the market with much longer periods of holding cash.

I'll give you one interesting example, Transmedics, this was a company that was 100% plus grower. Really exciting medical technology, helping people transplant organs.

Clearly better than the standard of care for moving organs around, which is a bucket of ice.

It was doing really well, then decided to buy aircraft to basically own the whole transplant process, using their own technicians and their own transportation.

And there's definitely some logic around that. But what happened was they then took that and then ran with it… too far.

One of those hundred-page short reports came out.

We had done our own work. We spoke to experts who had very positive things to say about Transmedics and why they use it. They found some people that had negative things, but the main negative thing they said, the core of the allegations, on top of little things like the CEO driving around in convoys of black SUVs, was that there doesn’t seem to be a cap on the amount that they could charge the health system for that whole organ transportation process.

So, the accusation was that they would strong-arm the hospitals into paying not just $100,000 USD for the use of their device, then charge maybe another $40,000 USD for flights and transportation to use Transmedics end-to-end.

And there was no cap on that.

So, Transmedics was quite proudly saying ‘we charge, and then we get paid’. But perhaps they pushed it too far, made too many enemies, and that system and politics might change, in which case they have a lot of expensive planes.

We sold a good chunk around 2.4x, so 140% up on our entry, purely system based.

And then they had their first quarterly revenue decline, the stock sold off, we exited the rest of our position on our risk systems, and then the short report came out and it got smoked.

It dropped another 20-30%, so we protected that last part. So combined with our systematic profit-taking, we ended up making serious money in a stock that went down last year.

So, it's a good example of how that works in practice, but there are many others.

Celsius is an energy soft drink, I love it, drank it when I was in United States. Extremely popular. It’s refreshing like an orange juice and a very strong cup of coffee all at once.

But sales stopped growing and this was a red-hot Twitter stock, and it reversed entirely, it's down over 70%.

So, if you had a long-term view that Celsius is amazing, people love it, the growth numbers are great, I’m going to hold forever, you've been stuck in a severe drawdown, whereas we got out systematically relatively close to the top, and there were other examples in the second half of last year. I should add that it was a serious market performer before that and was a very simple story, so this approach allows you to monetize an exciting opportunity like that, knowing htat if trends unpredictably reverse, you have a clear exit on both sides - if it does extremely well or poorly.

Ellianna

So what were your biggest winners and losers of the year?

Michael Frazis

The biggest was actually in semiconductors and Nvidia where we realized a 4x profit and we closed it out around June.

Transmedics was also up there in terms of biggest contributors.

Clarity Pharmaceuticals was another, our average price was almost exactly two bucks and then it went straight to eight really fast. And then we sold some, not all of it, so that reversed halfway.

We had all this cash from those sales so were looking for other opportunities, and found Orthocell, which was rolling out in the United States that went up 2.5x.

We invested in EBR in their capital raising. That went up more than 2.5x in the second half and into this year.

There was also Syntara, which was one of our larger contributors.

We'd been invested for a while; it was one of those companies where nobody really knew.

You’d go out to the facility in Sydney and they're just so happy to see you and would show you around.

The interesting thing about Syntara is that they do their science in-house, they developed their own drugs.

In Australia, there are a lot of companies in the biotech space that repurpose drugs or reformulate existing drugs. And these companies typically go for orphan drug designation and get some patent protection there, but it's actually quite short.

And the length of the patent protection directly impacts the amount that anyone's going to pay for it, so it really affects value.

You can do all the work, get all the approvals, and then be challenged in the courts. I've actually seen that happen, at least once in a company that we owned.

A generics company sued, and the courts decided that the IP wasn't real and the patents didn’t hold.

That obviously completely wipes out the value and that's the business model of generics companies.

If you have a composition-of-matter patent, if you design the drug yourself, the IP position is much stronger.

And you don't have to necessarily publish and tell everybody what you're doing.

You can be strategic about the time at which you want to file that patent which can push out that expiration date.

So Syntara did that very well.

Ellianna

On the topic of Aussie biotech companies like Syntara which developed their own medicine and technology; so you believe these companies are more likely to be successful than ones you mentioned who repurpose existing medicines and technologies, and are more likely to be approved by the FDA?

Michael Frazis

There's a number of opportunistic entrepreneurs, I guess you could call them, who are licensing drugs that exist, reformulating them or testing them in a new condition.

The good news is that often they work, there's generally a lot of data already.

But most sophisticated investors don't really get behind them for the reasons I said.

They know that it could get the orphan drug designation, but that's not very long so there might not be significant terminal value.

And if there’s legal risk around IP it's less interesting to a potential buyer.

The key point about IP risk is it's generally only litigated after it's approved.

Noone’s going to challenge the IP of a company doing a Phase 3 in Australia.

It's only after the drug is approved.

That’s when the challenge comes.

That's why sophisticated investors don’t support those companies in the same way they would a company developing a novel drug.

Ellianna

We recently had a call with Curvebeam, what did you think of that?

Michael Frazis

It was one of those mixed calls.

I mean the exciting thing is that the sales have picked up again, they had record sales of their device, both through their own sales efforts and through Stryker.

A while ago they basically handed over sales responsibility to Stryker in the US, a giant in the space. Striker needed more data and really pushed out the timeline on delivering sales. We were expecting last year to be the year that they would get significant revenues, now it looks like at best it will be 2025.

It should be around March that the extra data they requested is received and reviewed. But they did get a long way towards breakeven this quarter.

The downside for them is they only have three plus quarters of cash.

They think they can reach breakeven later this year, but that's cutting it too fine. And Aussie fundies want cash flow and profits anyway, not breakevens.

My view is they'll probably have to raise money.

I think it's a real opportunity if they do. That would remove the capital overhang on the stock, and they’ve demonstrated that they can get these sales. They're really just scratching the surface of their potential market.

We've done multiple channel checks with end buyers. Everybody's extremely positive about the demand for the product.

They just need to do these additional studies.

I'm sure they would agree that it was a mistake to wind back their own sales efforts, and I’ve certainly shared my opinion.

So this is a pivotal year for them.

It's hard with these illiquid companies that are only trading tens of thousands of dollars a day, because it really doesn't take much to move it one way or another.

The big holders invariably hold. Like you can't sell $5 million of stock in this company, whether you want to or not. You can't read too much into market movements. And medical device companies trade on rich multiples on success, and the technical founder has done it before.

But I do think that it's a company that will raise. When it does raise, that will probably be the moment for a turnaround.

There's another company we invested in, Anteris announced a dual listing on the Nasdaq and a huge raise, and ended up raising US$88 million.

That announcement cut the stock in half. Because why on earth would you buy a stock if you can just get loaded up in a raise? It just doesn't make sense.

If anything, you should short those companies into the raise, and then put more money into them then. If the capital requirement is large, you will get filled whether the company wants to reward a short seller or not. We're not shorting these companies, but it does come into play.

So, I think there'll probably be an overhang in Curvebeam until that's sorted.

The best-case scenario, they do some kind of strategic deal and they increase cash in a way that’s non-dilutive.

The company is extremely optimistic about their sales efforts this quarter. They did $5 million of purchase orders this quarter, and their cost base is $17 million a year.

So, they're pretty close to break even.

They need to make the inventory and then ship it, so there is a lot of capital tied up in that. They're trying to release that by running down their existing inventory.

If there’s a capital raise and sales continue to grow, then it could be one of those top performers, one of those companies that's kind of on the nose and then all of sudden doubles or triples quite quickly.

So, that's an exciting opportunity that we're invested in and have been for some time, this is a big year for them.

Ellianna

Absolutely, and it sounds like a really interesting company to look out for in the next year or so.

Which reminds me that I wanted to ask what your outlook is for 2025.

What are your predictions? What do you expect?

Michael Frazis

Well, the United States is in an interesting place because there's a huge amount of action going on at the top. It's quite exciting to see.

They're offering redundancies to every federal employee. This will, at the margin, reduce inflation. That's a lot of extra employees in the job market that will have to find something useful to do.

In Australia, inflation numbers are improving. We're still above 3%, but only just above and falling.

It seems like a lot of people are under a lot of pressure and you can see that in the data. There's been real estate falls across the market and it's just something you can feel. It's just a time where it feels like everybody is quite tight.

A lot of people did extremely well in real estate. But they're now paying far more interest than they expected a few years ago. Expenses are far higher across the board, incomes are not. Everybody's worried.

Making money on housing is great, but people are worried. They may be a beneficiary of high house prices, but they’ll also be worried about their children, how they're going to get on the market and the money that they're going to need just to buy a home. GDP per capita is falling fast in Australia.

Maybe if inflation comes down and rates are cut that starts to reverse.

In semiconductors, we’ll just have to see in 2025. There's only a handful of people making the core decisions, right? At the moment, they all want to win.

Sure DeepSeek come out with a cheap free model. Does that make these US founder-CEOs more or less competitive?

My guess is that if anything, it’s going to spur them on to win.

I'm still quite optimistic on that sector, but we have to manage it really carefully on risk. So last Monday, our entire semiconductor position basically went at or below the point at which we would close it out and wait for the next upswing (note - we’ve reduced positions since then).

I think that's the right way to invest in sectors like semiconductors. There’s going to be 1-3 year cycles where there’s moments of exuberance and demand surprises to the upside. Large language models are still not used at all by most people.

To go back to that example, if you can get a good AI sales agent that works well at the junior level, think about the sales function for most organizations.

There's somebody sending out a ton of emails, organizing meetings and writing papers and blogs that show how the product can be used.

That's a high paying high skilled job that an agent could do extremely well. And once you do, why have only one agent? Why not spin up a large number covering the whole market?

If you can afford 10 salespeople now, with AI agents, you can afford vastly more.

Which has brought us back to that software point.

It's probably a really good year for software, it’s pretty exciting.

It’s also probably going to be a year where risk management determines outcomes again.

And as always there will be individual growth opportunities that we'll have to find. We think we've got a couple in the portfolio that we're really excited about that we haven’t talked about yet.

Ellianna

Great, why don't we wrap up here then?

Michael Frazis

Sounds good. Thanks so much, Ellianna