One thing I like about Monish Pabrai is that he is willing to change his investment process if he believes that what he is using isn't optimal. We have seen it with his reduced concentration post-2008 and we have seen how he has adopted a comprehensive checklist to help avoid future mistakes.
At this year's Pabrai Funds AGM he gave an update on his checklist progress, and the following are notes from his update:
I talked in the last meeting about the checklist. I talked about how I found the checklist was important and profound in terms of its simplicity, and how wonderful it was in terms of the positive impacts on fund performance. I don’t want to go back to what we’ve talked about before. The history on the checklist is on the website, so if you’re seeing this for the first time, please go on the website and look at what we’ve talked about previously.
I’ve continued to work on the checklist and had three summer interns work on taking the checklist concept further. They were wonderful. In fact, for one of them, I decided to give him a free ad and I put his name at the bottom of the slide, “Joe” makes-Excel-dance “Benevento.”
Joe finished his first year at Harvard Business School and managed the other two interns over the summer. Basically, they analyzed 13F filings for a bunch of well-known, highly respected valued investors. You have to file a 13F every quarter if you run over $100 million of capital in the U.S. markets.
From the period of 2004 to 2010 we scanned all the 13F filings and all the public filings for Davis Funds, Oakmark, Third Avenue, Longleaf, Fairholme, Baupost, Greenlight, ESL, Pershing, Brave Warrior, Oak Value and Wintergreen.
If you track the 13F filings quarter after quarter, you can see changes that are made in the portfolios. We looked at where the company added a brand new position in a quarter which wasn’t in the 13F last quarter. We then looked at the next quarter to see if there was a change in the number of shares. If the number of shares went up, then we knew they continued buying.
In our analysis, we assumed that they were buying evenly throughout the quarter and came up with an average price. This probably isn’t reality, they probably had more sporadic buying, but I don’t think those deltas are very large.
We focused on the investments where the company had completely exited the investment. They held the investment, then at some point they completely sold the investment, and the investment was no longer on the 13F filing.
During the six-year period, we went through and extracted all the data about the gains and losses over all the positions that these funds held. We could clearly see the ones that they made money on. We could also see the ones that they lost money on, and approximately how much money they made or lost. On that list of mistakes, which is a long list of mistakes, we found 363 mistakes. What I mean by mistakes is there were 363 distinct different investments where this group of investors lost money. In aggregate, they lost about $20 billion from 2004-2010.
The largest mistake, Davis Funds, was $2 billion on their AIG (AIG, Financial) position. Of course, during the financial crisis there were large losses among financial firms, like Citigroup (C, Financial), Freddie Mae, Fannie Mae.
Because we had limited time in the summer, we picked about 25 of these mistakes. They were a diversified group of large mistakes in different industries made by different funds. Then we went back and looked at any commentary that we could find by either these investors or by others which would give us an idea of what caused them to make the investment. What were they thinking when they made the investment? How did that thinking change over the time that they held that position? When they exited, was anything said after they exited?
For example, we looked at AIG when Davis Funds made its investment in 2004. In their 2005 and 2006 Letters to Shareholders, they were talking about the investment. Hank Greenberg left the company in 2005 and the new CEO started in 2006. In 2006, they were talking about that transition. We looked at the AIG stock price during this period, and how the stock price was moving versus the commentary they were making. All the way down to when the crisis hit and the feds had to come and bail out AIG, we could see how the commentary was changing.
So from all that, we looked at why they made the investment, and what they perceived was the mistake they made. We looked at if they talked about the stock after they exited. We also evaluated it ourselves, and thought about what caused the mistake.
Another example, is Longleaf Partners with their investment in General Motors (GM). They had a certain viewpoint on GM when they were hot and heavy buying GM stock in 2006 and 2007.
Finally, when they exited GM at an almost complete loss, they talked about what went wrong. We basically extracted all that data and then analyzed it. I’m still going through that process right now and I have added questions to my checklist.
Additionally, we looked at Baupost. There are mistakes made by Seth Klarman where he loses money. But Baupost never talks about the mistakes. If you look at all the Baupost letters, which are hard to get, there isn’t any commentary on any of the holdings, and especially not on any of the mistakes.
For example, Baupost made an in investment in Sally Mae. After about six weeks, there was a buyout
offer for the firm and they had a 30% increase from that buy-out offer. But then, about six months later the buy-out blew up because the buyers wouldn’t close on the transaction due to a lot of issues coming into Sallie Mae. The stock nose-dived well below Baupost’s buy price. After the nose dive, Baupost loaded up and added significantly to the position. About a quarter later, they exited with a loss.
The whole checklist exercise is similar to what the FAA does in plane crash investigations. When a plane crash occurs, the FAA goes in, and figures out exactly what happened. After they figure out what happened, they make changes to airline safety which will decrease the odds of that particular crash with those circumstances. That’s exactly what we are doing with this exercise.
It was a good exercise and a lot fun to see the data. It was quite eye opening. Like I said, the interns did a great job. More power to them.
You can see that we basically scratched the surface. We looked at 363 mistakes, and I was able to reverse engineer about 30 of them. We haven’t touched the other 330 mistakes. We haven’t touched mistakes by a whole bunch of other investors and we haven’t touched mistakes in other periods. This study was for 2004-2010. I have started another study which is for 1994-2004, and this will give me another data set. The interns had a little bit of time so we went through and we extracted data for the Longleaf mistakes from 1998-2004. Again, I’m still going through and analyzing this data.
I’m extracting the lessons which I will be adding to my checklist. It’s a great process, but is endless in the sense that I don’t think we’ll ever get to the point where we’re done with checklist items. It’s been about 2 years since I started working with the checklist, and we’ve had 18 new investments over this two-year period in the post checklist era. We’ve had three full exits, all of which have been profitable. It’s highly likely that we’ll not have any mistakes amongst these 18 investments. This could change but at least that’s the best I can tell today.
Some of these bets that we made were made as basket bets where I saw a somewhat elevated risk of a loss, but I also saw an asymmetrical return where the returns were extremely lop-sided.
There was an 80-90% probability that we could make a 5x or 10x return, and a 5-10% probability that we might have a loss of capital.
When I encountered those types of investments I would put a small bet on it, typically two percent of capital. I made a number of bets of this type and grouped it as a basket. In this basket, I expected that some of them may not go our way. But so far, what’s happened is that 100 percent of the basket has worked fine. I don’t expect this going forward. I will absolutely have mistakes, but I expect the error rate to go down.
The best way to learn is to learn from other people’s mistakes. My first checklist was based on my mistakes. Then, I added other mistakes, like Charlie Munger and Warren Buffett mistakes.
The good thing about Warren and Charlie is that they are very transparent about their mistakes over the decades. There is a very rich commentary about the investments that didn’t work out, and what happened.
At this year's Pabrai Funds AGM he gave an update on his checklist progress, and the following are notes from his update:
I talked in the last meeting about the checklist. I talked about how I found the checklist was important and profound in terms of its simplicity, and how wonderful it was in terms of the positive impacts on fund performance. I don’t want to go back to what we’ve talked about before. The history on the checklist is on the website, so if you’re seeing this for the first time, please go on the website and look at what we’ve talked about previously.
I’ve continued to work on the checklist and had three summer interns work on taking the checklist concept further. They were wonderful. In fact, for one of them, I decided to give him a free ad and I put his name at the bottom of the slide, “Joe” makes-Excel-dance “Benevento.”
Joe finished his first year at Harvard Business School and managed the other two interns over the summer. Basically, they analyzed 13F filings for a bunch of well-known, highly respected valued investors. You have to file a 13F every quarter if you run over $100 million of capital in the U.S. markets.
From the period of 2004 to 2010 we scanned all the 13F filings and all the public filings for Davis Funds, Oakmark, Third Avenue, Longleaf, Fairholme, Baupost, Greenlight, ESL, Pershing, Brave Warrior, Oak Value and Wintergreen.
If you track the 13F filings quarter after quarter, you can see changes that are made in the portfolios. We looked at where the company added a brand new position in a quarter which wasn’t in the 13F last quarter. We then looked at the next quarter to see if there was a change in the number of shares. If the number of shares went up, then we knew they continued buying.
In our analysis, we assumed that they were buying evenly throughout the quarter and came up with an average price. This probably isn’t reality, they probably had more sporadic buying, but I don’t think those deltas are very large.
We focused on the investments where the company had completely exited the investment. They held the investment, then at some point they completely sold the investment, and the investment was no longer on the 13F filing.
During the six-year period, we went through and extracted all the data about the gains and losses over all the positions that these funds held. We could clearly see the ones that they made money on. We could also see the ones that they lost money on, and approximately how much money they made or lost. On that list of mistakes, which is a long list of mistakes, we found 363 mistakes. What I mean by mistakes is there were 363 distinct different investments where this group of investors lost money. In aggregate, they lost about $20 billion from 2004-2010.
The largest mistake, Davis Funds, was $2 billion on their AIG (AIG, Financial) position. Of course, during the financial crisis there were large losses among financial firms, like Citigroup (C, Financial), Freddie Mae, Fannie Mae.
Because we had limited time in the summer, we picked about 25 of these mistakes. They were a diversified group of large mistakes in different industries made by different funds. Then we went back and looked at any commentary that we could find by either these investors or by others which would give us an idea of what caused them to make the investment. What were they thinking when they made the investment? How did that thinking change over the time that they held that position? When they exited, was anything said after they exited?
For example, we looked at AIG when Davis Funds made its investment in 2004. In their 2005 and 2006 Letters to Shareholders, they were talking about the investment. Hank Greenberg left the company in 2005 and the new CEO started in 2006. In 2006, they were talking about that transition. We looked at the AIG stock price during this period, and how the stock price was moving versus the commentary they were making. All the way down to when the crisis hit and the feds had to come and bail out AIG, we could see how the commentary was changing.
So from all that, we looked at why they made the investment, and what they perceived was the mistake they made. We looked at if they talked about the stock after they exited. We also evaluated it ourselves, and thought about what caused the mistake.
Another example, is Longleaf Partners with their investment in General Motors (GM). They had a certain viewpoint on GM when they were hot and heavy buying GM stock in 2006 and 2007.
Finally, when they exited GM at an almost complete loss, they talked about what went wrong. We basically extracted all that data and then analyzed it. I’m still going through that process right now and I have added questions to my checklist.
Additionally, we looked at Baupost. There are mistakes made by Seth Klarman where he loses money. But Baupost never talks about the mistakes. If you look at all the Baupost letters, which are hard to get, there isn’t any commentary on any of the holdings, and especially not on any of the mistakes.
For example, Baupost made an in investment in Sally Mae. After about six weeks, there was a buyout
offer for the firm and they had a 30% increase from that buy-out offer. But then, about six months later the buy-out blew up because the buyers wouldn’t close on the transaction due to a lot of issues coming into Sallie Mae. The stock nose-dived well below Baupost’s buy price. After the nose dive, Baupost loaded up and added significantly to the position. About a quarter later, they exited with a loss.
The whole checklist exercise is similar to what the FAA does in plane crash investigations. When a plane crash occurs, the FAA goes in, and figures out exactly what happened. After they figure out what happened, they make changes to airline safety which will decrease the odds of that particular crash with those circumstances. That’s exactly what we are doing with this exercise.
It was a good exercise and a lot fun to see the data. It was quite eye opening. Like I said, the interns did a great job. More power to them.
You can see that we basically scratched the surface. We looked at 363 mistakes, and I was able to reverse engineer about 30 of them. We haven’t touched the other 330 mistakes. We haven’t touched mistakes by a whole bunch of other investors and we haven’t touched mistakes in other periods. This study was for 2004-2010. I have started another study which is for 1994-2004, and this will give me another data set. The interns had a little bit of time so we went through and we extracted data for the Longleaf mistakes from 1998-2004. Again, I’m still going through and analyzing this data.
I’m extracting the lessons which I will be adding to my checklist. It’s a great process, but is endless in the sense that I don’t think we’ll ever get to the point where we’re done with checklist items. It’s been about 2 years since I started working with the checklist, and we’ve had 18 new investments over this two-year period in the post checklist era. We’ve had three full exits, all of which have been profitable. It’s highly likely that we’ll not have any mistakes amongst these 18 investments. This could change but at least that’s the best I can tell today.
Some of these bets that we made were made as basket bets where I saw a somewhat elevated risk of a loss, but I also saw an asymmetrical return where the returns were extremely lop-sided.
There was an 80-90% probability that we could make a 5x or 10x return, and a 5-10% probability that we might have a loss of capital.
When I encountered those types of investments I would put a small bet on it, typically two percent of capital. I made a number of bets of this type and grouped it as a basket. In this basket, I expected that some of them may not go our way. But so far, what’s happened is that 100 percent of the basket has worked fine. I don’t expect this going forward. I will absolutely have mistakes, but I expect the error rate to go down.
The best way to learn is to learn from other people’s mistakes. My first checklist was based on my mistakes. Then, I added other mistakes, like Charlie Munger and Warren Buffett mistakes.
The good thing about Warren and Charlie is that they are very transparent about their mistakes over the decades. There is a very rich commentary about the investments that didn’t work out, and what happened.
