Spring 2021 webinar transcript
Hello, I'm Michael Robbins. I teach several courses in quantitative asset management to graduate students at Columbia University and writing a book, Quantitative Asset Management for McGraw-Hill to be published in 2022. The website for the book is quantitativeassetmanagement.com, where there will be quite a bit of MATLAB code, which we'll discuss today, and is also discussed in depth in the book. And you can ask questions about today's presentation at info@quantitativeassetmanagement.com.
I brought two students with me today to help describe how we use MATLAB's backtesting framework in the courses. I've been the chief investment officer of five large investment firms and the chief investment risk officer of a large pension system. And I've taught for Columbia University for quite a few years now.
Malin has taken my environmental social and governance course and has been a CA in my global tactical allocations course. Yao took my external managers course and is now a CA in that course. We won't be able to discuss the courses in detail. We'll provide a high-level overview of some of the slides. If you want to learn more about the courses, you can read my book. I go into them in quite a bit of detail.
Today, we'll talk about an overview of how we modified MATLAB's code and the ESG and GTAA courses, as well as the external managers course. The programming for the course has all followed the same general framework, where we gather a lot of proprietary data. We organize it in the database. We perform machine learning on that data as inputs into the backtesting framework. And then after running the backtester, we use performance attribution to study the results.
I'll talk about how we modified the code. Malin will talk about ESG and GTAA. And Yao will talk about external managers.
From the very beginning, it's been very important to us to use vectorized inputs in the backtesting framework and not just the event-driven code. This is for a few reasons. Very importantly, most of the students use the language of their choice to do the factor engineering and provide that as an input, rather than programming it into the rebalance function. It also allowed us to use technologies that were cumbersome to use inside the backtesting framework, or just took too long to calculate in that framework.
Technologies, like MATLAB's Classification Learner app, Amazon's Prophet, and Facebook's PyTouch. It also allowed us to modify the code to use optional input data, like volume and open interest data, to be used for things like calculating market impact. We modified the code to pass that optional data directly into the cost and rebalance functions to use in sophisticated calculations.
The backtesting engine runs in a loop and calls the rebalance and cost functions at each rebalance iteration. The rebalance function creates buy and sell requests to be evaluated by the cost function. Looking at the price of the day, the rebalance function might, for instance, say buy 100 shares of ABC. The rebalance function is past that data, along with the position sizes. A modification we made to the backtesting code.
The cost function then evaluates the market impact and transaction fees. It also decides how much of the transaction can be executed. For instance, it might decide that the price of the purchase should be $25 instead of $20. And that the full 100 share lot is not reasonable to execute given, for instance, the volume of the day. It passes that information back to the backtesting engine, a modification we made to the code.
It might say, for instance, you cannot transact 100 shares. You have to only transact 90. We've modified the backtesting engine to transact only amount that the cost function allows and not the amount that the rebalance function requested. We also modified it to perform other calculations that were very important, including fee structures that were complicated by path dependency.
We included management fees, incentive fees, hurdle rates, and high watermarks, as well as other complications that we needed in order to do our analysis. In an actual fund, it's important to include all these things in your simulations because they need to be provided to the clients to show the performance of the fund. The fund's performance is not useful without these intricate calculations.
We also included an overdraft account that used time varying lending rates. This was very important for two reasons. If we wanted to make purchases that exceeded the amount of money in our cash account, we wanted to penalize the rebalance function for not setting aside enough cash by charging the lending rate. We also did not want to adjust the percentages of the other investments automatically. We wanted the rebalance function to explicitly request sales of other assets.
So instead of automatically reducing the percentages of other assets, we borrowed money from the overdraft account. This also allowed us to use leverage in our portfolios, which is very important for a number of portfolio managers. It allowed us to purchase more assets than we had cash for by borrowing money at the lending rate. This lending rate continued to be used until the rebalance function explicitly paid back the overdraft account, allowing us to have persistent leverage.
Another important adjustment that we made to the backtesting framework is to provide independent profit and loss streams for each asset. This provided a number of additional abilities that the backtesting engine didn't have, including being able to analyze those performance streams in the rebalance function to do things like stop losses. And also, very importantly, to do asset-level analysis like Brinson performance attribution after the backtesting analysis was done.
Using all these adjustments that we made to the rebalance engine, we were able to integrate with the MATLAB statistics and machine learning toolbox and the deep learning toolbox to make the inputs. The data feed toolbox, in order to actually transact with companies like interactive brokers. The transaction cost analysis tool box in the cost function in order to do sophisticated market impact analysis. And to use a multi-period Brinson performance attribution at the end. We ran all these what-if analyses in a large loop and produced thousands of outputs to be analyzed later.
Now, Malin will discuss how we used these analyses in the ESG and GTAA classes.
Thank you, Michael. So over the past year or so, I have worked on a wide variety of projects together with Michael. And the very first one was an ESG analyses.
This was for a fund with 193 employees in seven offices. So this is a sophisticated fund who specialized in ESG factors. And they provided us with the proprietary data, as well as their returns. And throughout the project, the team and I used machine learning and backtesting to evaluate the factors and identify any potential areas for improvement.
So the first thing we had to do was to make sure that the real performance was reconciled with the simulations. So in order to analyze our data, we needed to compare the actual performance of investment portfolios to the simulated performance of those portfolios under different conditions. So for example, with or without tax harvesting. So we took the original allocations, and we used that as an input and ran through the backtester to make sure that we got the same results as in reality.
And of course, in order to do this, we needed software. So this is where MathWorks came in. And they built a really excellent tool for us. However, it lacked a few features that we needed.
So the first requirement was that we could use the vectorized input to prove out the original performance before simulating the what-ifs. So after completing this, we moved on. And we used the triple-barrier method to build our response set and walk-forward analysis to evaluate our strategies. We also used the Classification Learner app to train models on the data and to hyper-parameterize the models.
For the walk-forward analyses, we use quarantine gaps between the training and the test set. And we did this in order to minimize the possibility of auto-correlation between these two periods. And both the backtester and the look-forward helped us realize a pattern of decreasing efficacy. So this was one of many, many ways that the backtester proved incredibly helpful for the team, as we were able to identify this behavior.
We also explored risk control overlays throughout the project. And after wrapping up the ESG project, we moved on to look at global macroeconomic factors. And for this, we worked in different teams. And we were competing for the model predictive models.
So looking at the tree chart here, each node represents a team. And that team had a decision for which out of two assets to buy or sell. So looking at the top left corner here, for example, commodities versus equities, the team there had to run models to decide on whether to buy or sell commodities or equities. At the end of the project, we combined all of these results, and we built a global tactical asset allocation strategy. And for this, we used the same methods that we had for the ESG project.
And last, but definitely not least, Michael built an app on the Bloomberg terminals on campus from which students can directly download data from Bloomberg and run different machine learning algorithms. And with that, I will pass it over to Yao.
Next, I will talk about the analysis we did on the data of 101 external managers. A data company generously offered a rich and comprehensive database of global advisor holdings and performance to provide an in-depth look inside the portfolios. They produced this by funding accounts with robo-advisors to track the performance on those accounts.
And they asked us several questions. We answered those questions using the data they provided. The first question they have is, how do the fund compare to each other? For instance, compare all the taxable funds to each other, all the social investment funds to each other.
The second question they have is, how all the funds' performance change if the rebalance frequency was altered. For instance, if a dividend investing fund were to wait until their rebalance date to invest or if a tax-loss harvesting fund were to wait until their rebalance date? And the third question is, how would funds' performance change if their composition were altered? For example, what happens if large cash allocations were reduced to be similar to that of their peers on a risk adjusted basis?
To answer these questions, we need to reconcile the real performance with the simulated results. We use the vectorized input, and that is the signal table, to provide the original performance before simulating our what-if questions. To accomplish this, we used the actual fund asset allocation as the signal file and the rebalance function. And rebalanced the portfolio to the signal weights. We also performed benchmark analysis. We reverse engineered a policy portfolio, a benchmark, and used those allocations in a signal file to simulate the benchmark returns and attributions.
One of the enhancements we added is to split the return stream for attribution in stop-loss analysis. The original backtester provided returns and costs for the portfolio as a whole, but we needed to divide these return streams and record them for each asset. This was required for the final multi-period Brinson attribution analysis and also for some rebalance logic, like stop-loss orders.
As easy as it may sound, it is not a trivial task. Besides splitting the return stream, we also added a second cash account. And we call that the overdraft account. The original backtester didn't distinguish between lending and borrowing. And that's why we need an overdraft account.
This allowed us to transact when the cash was used up without selling other stocks to generate cash. Also, this allowed us to stimulate levered funds. Like, if we want to borrow 20% of the assets in the fund, we'll have a 120 to 20 structure. Another enhancement is we added complex cost and fee structures, including incentive fees, management fees, crystallization dates, hurdle rates, and high watermarks.
This complex cost structure enabled us to do more interesting analysis. For example, we can do past dependant fees and costs for simulations. Also, we can perform multi-period Brinson for performance and risk attribution analysis. Here's a comparison between the summary function of the original backtester and the enhanced backtester. The one on the left is limited to the portfolio level results. And the enhanced backtester can disaggregated the results using the multi-period Brinson method.
With all the great features that we added, here are still a lot on our to-do list. And here are several questions that we want to answer. The first one is, how did each portfolio perform outright and compared to its benchmark using a variety of portfolio-level measures and fund-level Brinson analysis? The second question is, how did portfolios compare to each other in groups? For instance, all social fund outright versus each other and relative to their benchmarks.
The third one is, how did groups compare? For instance, the average social funds versus the average taxable funds, a particular manager's social fund versus his taxable fund outright, the social performance versus their benchmark. The fourth question we want to answer is, how did portfolios compare a risk-adjusted basis? For instance, use CAPM to leverage the portfolios to an equal level. Adjust the equity to a fixed income ratio to equalize risk and simulate the results.
And the last question is, was tax-loss harvesting actually effective? For example, compare a manager's tax-loss harvesting funds to their taxable fund. And again, after eliminating the tax-loss harvesting rebalancing date. Also do this after estimating taxes as well.
And that concludes our presentation. Thank you so much for listening. If you have any questions, please send us emails. And if you want more information, please visit the website. All of the contact information is on the upper right corner.
I brought two students with me today to help describe how we use MATLAB's backtesting framework in the courses. I've been the chief investment officer of five large investment firms and the chief investment risk officer of a large pension system. And I've taught for Columbia University for quite a few years now.
Malin has taken my environmental social and governance course and has been a CA in my global tactical allocations course. Yao took my external managers course and is now a CA in that course. We won't be able to discuss the courses in detail. We'll provide a high-level overview of some of the slides. If you want to learn more about the courses, you can read my book. I go into them in quite a bit of detail.
Today, we'll talk about an overview of how we modified MATLAB's code and the ESG and GTAA courses, as well as the external managers course. The programming for the course has all followed the same general framework, where we gather a lot of proprietary data. We organize it in the database. We perform machine learning on that data as inputs into the backtesting framework. And then after running the backtester, we use performance attribution to study the results.
I'll talk about how we modified the code. Malin will talk about ESG and GTAA. And Yao will talk about external managers.
From the very beginning, it's been very important to us to use vectorized inputs in the backtesting framework and not just the event-driven code. This is for a few reasons. Very importantly, most of the students use the language of their choice to do the factor engineering and provide that as an input, rather than programming it into the rebalance function. It also allowed us to use technologies that were cumbersome to use inside the backtesting framework, or just took too long to calculate in that framework.
Technologies, like MATLAB's Classification Learner app, Amazon's Prophet, and Facebook's PyTouch. It also allowed us to modify the code to use optional input data, like volume and open interest data, to be used for things like calculating market impact. We modified the code to pass that optional data directly into the cost and rebalance functions to use in sophisticated calculations.
The backtesting engine runs in a loop and calls the rebalance and cost functions at each rebalance iteration. The rebalance function creates buy and sell requests to be evaluated by the cost function. Looking at the price of the day, the rebalance function might, for instance, say buy 100 shares of ABC. The rebalance function is past that data, along with the position sizes. A modification we made to the backtesting code.
The cost function then evaluates the market impact and transaction fees. It also decides how much of the transaction can be executed. For instance, it might decide that the price of the purchase should be $25 instead of $20. And that the full 100 share lot is not reasonable to execute given, for instance, the volume of the day. It passes that information back to the backtesting engine, a modification we made to the code.
It might say, for instance, you cannot transact 100 shares. You have to only transact 90. We've modified the backtesting engine to transact only amount that the cost function allows and not the amount that the rebalance function requested. We also modified it to perform other calculations that were very important, including fee structures that were complicated by path dependency.
We included management fees, incentive fees, hurdle rates, and high watermarks, as well as other complications that we needed in order to do our analysis. In an actual fund, it's important to include all these things in your simulations because they need to be provided to the clients to show the performance of the fund. The fund's performance is not useful without these intricate calculations.
We also included an overdraft account that used time varying lending rates. This was very important for two reasons. If we wanted to make purchases that exceeded the amount of money in our cash account, we wanted to penalize the rebalance function for not setting aside enough cash by charging the lending rate. We also did not want to adjust the percentages of the other investments automatically. We wanted the rebalance function to explicitly request sales of other assets.
So instead of automatically reducing the percentages of other assets, we borrowed money from the overdraft account. This also allowed us to use leverage in our portfolios, which is very important for a number of portfolio managers. It allowed us to purchase more assets than we had cash for by borrowing money at the lending rate. This lending rate continued to be used until the rebalance function explicitly paid back the overdraft account, allowing us to have persistent leverage.
Another important adjustment that we made to the backtesting framework is to provide independent profit and loss streams for each asset. This provided a number of additional abilities that the backtesting engine didn't have, including being able to analyze those performance streams in the rebalance function to do things like stop losses. And also, very importantly, to do asset-level analysis like Brinson performance attribution after the backtesting analysis was done.
Using all these adjustments that we made to the rebalance engine, we were able to integrate with the MATLAB statistics and machine learning toolbox and the deep learning toolbox to make the inputs. The data feed toolbox, in order to actually transact with companies like interactive brokers. The transaction cost analysis tool box in the cost function in order to do sophisticated market impact analysis. And to use a multi-period Brinson performance attribution at the end. We ran all these what-if analyses in a large loop and produced thousands of outputs to be analyzed later.
Now, Malin will discuss how we used these analyses in the ESG and GTAA classes.
Thank you, Michael. So over the past year or so, I have worked on a wide variety of projects together with Michael. And the very first one was an ESG analyses.
This was for a fund with 193 employees in seven offices. So this is a sophisticated fund who specialized in ESG factors. And they provided us with the proprietary data, as well as their returns. And throughout the project, the team and I used machine learning and backtesting to evaluate the factors and identify any potential areas for improvement.
So the first thing we had to do was to make sure that the real performance was reconciled with the simulations. So in order to analyze our data, we needed to compare the actual performance of investment portfolios to the simulated performance of those portfolios under different conditions. So for example, with or without tax harvesting. So we took the original allocations, and we used that as an input and ran through the backtester to make sure that we got the same results as in reality.
And of course, in order to do this, we needed software. So this is where MathWorks came in. And they built a really excellent tool for us. However, it lacked a few features that we needed.
So the first requirement was that we could use the vectorized input to prove out the original performance before simulating the what-ifs. So after completing this, we moved on. And we used the triple-barrier method to build our response set and walk-forward analysis to evaluate our strategies. We also used the Classification Learner app to train models on the data and to hyper-parameterize the models.
For the walk-forward analyses, we use quarantine gaps between the training and the test set. And we did this in order to minimize the possibility of auto-correlation between these two periods. And both the backtester and the look-forward helped us realize a pattern of decreasing efficacy. So this was one of many, many ways that the backtester proved incredibly helpful for the team, as we were able to identify this behavior.
We also explored risk control overlays throughout the project. And after wrapping up the ESG project, we moved on to look at global macroeconomic factors. And for this, we worked in different teams. And we were competing for the model predictive models.
So looking at the tree chart here, each node represents a team. And that team had a decision for which out of two assets to buy or sell. So looking at the top left corner here, for example, commodities versus equities, the team there had to run models to decide on whether to buy or sell commodities or equities. At the end of the project, we combined all of these results, and we built a global tactical asset allocation strategy. And for this, we used the same methods that we had for the ESG project.
And last, but definitely not least, Michael built an app on the Bloomberg terminals on campus from which students can directly download data from Bloomberg and run different machine learning algorithms. And with that, I will pass it over to Yao.
Next, I will talk about the analysis we did on the data of 101 external managers. A data company generously offered a rich and comprehensive database of global advisor holdings and performance to provide an in-depth look inside the portfolios. They produced this by funding accounts with robo-advisors to track the performance on those accounts.
And they asked us several questions. We answered those questions using the data they provided. The first question they have is, how do the fund compare to each other? For instance, compare all the taxable funds to each other, all the social investment funds to each other.
The second question they have is, how all the funds' performance change if the rebalance frequency was altered. For instance, if a dividend investing fund were to wait until their rebalance date to invest or if a tax-loss harvesting fund were to wait until their rebalance date? And the third question is, how would funds' performance change if their composition were altered? For example, what happens if large cash allocations were reduced to be similar to that of their peers on a risk adjusted basis?
To answer these questions, we need to reconcile the real performance with the simulated results. We use the vectorized input, and that is the signal table, to provide the original performance before simulating our what-if questions. To accomplish this, we used the actual fund asset allocation as the signal file and the rebalance function. And rebalanced the portfolio to the signal weights. We also performed benchmark analysis. We reverse engineered a policy portfolio, a benchmark, and used those allocations in a signal file to simulate the benchmark returns and attributions.
One of the enhancements we added is to split the return stream for attribution in stop-loss analysis. The original backtester provided returns and costs for the portfolio as a whole, but we needed to divide these return streams and record them for each asset. This was required for the final multi-period Brinson attribution analysis and also for some rebalance logic, like stop-loss orders.
As easy as it may sound, it is not a trivial task. Besides splitting the return stream, we also added a second cash account. And we call that the overdraft account. The original backtester didn't distinguish between lending and borrowing. And that's why we need an overdraft account.
This allowed us to transact when the cash was used up without selling other stocks to generate cash. Also, this allowed us to stimulate levered funds. Like, if we want to borrow 20% of the assets in the fund, we'll have a 120 to 20 structure. Another enhancement is we added complex cost and fee structures, including incentive fees, management fees, crystallization dates, hurdle rates, and high watermarks.
This complex cost structure enabled us to do more interesting analysis. For example, we can do past dependant fees and costs for simulations. Also, we can perform multi-period Brinson for performance and risk attribution analysis. Here's a comparison between the summary function of the original backtester and the enhanced backtester. The one on the left is limited to the portfolio level results. And the enhanced backtester can disaggregated the results using the multi-period Brinson method.
With all the great features that we added, here are still a lot on our to-do list. And here are several questions that we want to answer. The first one is, how did each portfolio perform outright and compared to its benchmark using a variety of portfolio-level measures and fund-level Brinson analysis? The second question is, how did portfolios compare to each other in groups? For instance, all social fund outright versus each other and relative to their benchmarks.
The third one is, how did groups compare? For instance, the average social funds versus the average taxable funds, a particular manager's social fund versus his taxable fund outright, the social performance versus their benchmark. The fourth question we want to answer is, how did portfolios compare a risk-adjusted basis? For instance, use CAPM to leverage the portfolios to an equal level. Adjust the equity to a fixed income ratio to equalize risk and simulate the results.
And the last question is, was tax-loss harvesting actually effective? For example, compare a manager's tax-loss harvesting funds to their taxable fund. And again, after eliminating the tax-loss harvesting rebalancing date. Also do this after estimating taxes as well.
And that concludes our presentation. Thank you so much for listening. If you have any questions, please send us emails. And if you want more information, please visit the website. All of the contact information is on the upper right corner.