By Dominik Peters
Created 2017-04-18. Revised 2017-05-19. We're centralising all discussion on the Effective Altruism forum. To discuss this post, please comment there.
We have attempted to build a quantitative model to estimate the impact of the Good Food Institute (GFI). We have found this exceptionally difficult due to the diversity of GFI’s activities and the particularly unclear counterfactuals. In this post, I explain some of the modelling approaches we tried, and why we are not satisfied with them. This post assumes good background knowledge about GFI, you can read more at Animal Charity Evaluators.
Approach 0: Direct estimation
Our first model of GFI involved directly estimating by how many years a fully funded GFI would accelerate the arrival of chicken product substitutes. Our first intuition was to put this at 5 years, but we realised that we had next to no intuitive grasp at all on this figure. So we attempted to find approaches that involved estimating quantities we have a better intuitive grasp on.
Approach 1: Multiplier on investment into animal substitutes
A donation of $1 to GFI increases the amount of investments (by VCs, government research councils, other grant-making institutions) by $X, through creating new investment opportunities (like start-ups) and by making the field more attractive in general.
For example, New Harvest was instrumental in starting companies working on yeast-based replacements for dairy and egg products, by introducing future founders to each other and giving them a small start-up grant (of about $30-50k each). These companies subsequently source additional investment of about $3m. However, we do not believe that this is very informative for estimating future multipliers, since New Harvest might have picked very low-hanging fruit in these instances.
Next, we would need some account of how this increased investment would have accelerated the development of meat alternatives (so as to produce value). For this, we would need to estimate when this investment of $X would otherwise have occurred, but it is unclear how to figure this out.
We did not come up with good strategies for breaking these difficulties down into smaller chunks that would be easier to model.
Approach 2: Direct Investment
A very simple modelling strategy involves estimating the rough amount of research effort (capital investments and research hours) that will be required in total and eventually to get to a “solution”, i.e., availability of attractive substitutes for animal products. One could obtain such an estimate by enumerating the list of animal products that need to be replaced, and then look at how much effort was needed to develop products such as the Impossible Burger. Next, one could assume that no-one else would ever invest in these opportunities. Then, by estimating the value of having substitutes and multiplying by fraction of the total effort required that our donation financed, we would get an estimate of the impact of our donation.
However, this approach is a bit silly because it does not model the acceleration of research: If there are no other donors in the field, then our donation is futile because £10,000 will not fund the entire effort required.
Approach 3: Acceleration Dynamics
How are we going to reach the stage at which attractive meat substitutes are widely available? Well, companies and other research groups will have to expend some amount of effort into the problem, and the more cumulative effort has been expended the closer we are to a good solution. Our donation to GFI could be modelled as an external “shock” to the amount of effort invested into the field from that point in time onwards. Graphically, this could look like this:
Whether the unperturbed curve is linear is unclear; it could be convex.
Now, with additional effort invested into the problem, we are getting closer to a solution, and in particular the quality of meat substitutes available increases. Again, it is not obvious how the quality of these products is functionally related to the amount of cumulative effort expended; one possible shape would be an S-curve (which increases rapidly after some initial breakthroughs have been achieved, and flattens out when perfecting things), or it could be a curve indicating diminishing returns throughout (if we think that increasing quality becomes harder and harder), or many other possible shapes (consisting of many separate discoveries), or exponential (like in Moore’s law). Different choices of shapes imply different magnitudes of impact, and we found no good way of figuring out which shape fits the particular situation.
We quickly became dissatisfied with each of the modelling approaches we tried. They either had major flaws (like failing to model acceleration dynamics) or did not succeed in actually breaking down our uncertainty into smaller, more manageable components.