Bridging Distant Ideas: the Impact of AI on R&D and Recombinant Innovation
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.
Ruth: I'm Ruth.
Warren: And I'm Warren, here with Iris.
Ruth: We're looking at the paper: "Bridging Distant Ideas".
Warren: Here's the gist of the paper. We study how artificial intelligence (AI) affects firms’ incentives to pursue incremental versus radical knowledge recombinations This paper develops a model of recombinant innovation embedded in a Schumpeterian quality-ladder framework to investigate the impact of AI on R&D and recombinant innovation.
Ruth: Model Framework The study develops a model of recombinant innovation embedded in a Schumpeterian quality-ladder framework, where innovation arises from recombining ideas across varying distances in a knowledge space.
Warren: This framework positions ideas in a graph structure where distances capture the cognitive dissimilarity between fields R&D firms choose how far to reach across this space, facing a fundamental trade-off between distant recombinations and close recombinations.
Ruth: Innovation arrives as a Poisson process with an arrival rate that decreases with recombinant distance, and each successful innovation grants the innovator a temporary monopoly until displaced by the next one.
Title and authors: Ruth: So we’re looking at this paper today called "Bridging Distant Ideas: the Impact of AI on R andD and Recombinant Innovation." It really zeroes in on how artificial intelligence affects whether companies try to make small, incremental changes or big, radical leaps in their research.
Warren: That sounds like it’s digging into the core tension there, Ruth. It’s about those incentives to pursue different types of knowledge recombinations.
Ruth: Exactly. The authors set up a model using a Schumpeterian quality ladder framework to see how ideas are recombined across different distances in a knowledge space, and then they map AI's influence onto that structure.
Iris: So the idea is that innovation isn't just making things better on the same topic, but actually connecting ideas from very different fields, which requires bridging those distances between concepts.
Ruth: Right. And they build this framework where distance in the knowledge space represents how dissimilar two ideas are cognitively, and firms choose how far they want to stretch across that space.
Warren: It sounds like they’re trying to balance the temptation to go super distant, which is radical innovation, against staying close by doing incremental work.
Iris: The model introduces a Poisson process for innovation arrivals where the rate of those new ideas actually decreases as you move further away from the initial idea, because it’s harder to find connections that far out.
Ruth: And this is where AI comes in as a factor that changes things. They look at four different ways AI can influence R andD incentives, and they separate them into positive and negative effects.
Warren: What are the positive effects they found? I assume some of these help firms innovate better, right?
Iris: The positive side shows that AI can actually increase the chance that a specific recombination attempt succeeds, which effectively shrinks that distance between ideas and lets firms try more ambitious research strategies.
Ruth: That suggests AI helps bridge gaps in knowledge space by making those distant connections more feasible. They also mention it can create new intermediate connections, which is like some diffusion effect you see with general purpose technologies.
Title and authors: Warren: So AI makes the long jumps easier to make, but that sounds like a double-edged sword because they immediately pivot to the negative stuff.
Iris: The negative effects involve AI empowering competitors by raising the overall rate of innovations across the entire economy, which speeds up creative destruction and shortens how long those innovation monopolies last.
Ruth: And they also point out things like the streetlight effect and stepping-on-toes effect, where AI might actually reduce the originality of research when firms rely on it too much.
Warren: So they’re arguing that while AI helps individual firms reach further, it simultaneously makes the whole competitive landscape more volatile and less rewarding for sticking to one big idea for a long time.
Iris: The authors get three main results from this model. First, they find that higher AI productivity encourages more distant recombinations if the direct facilitation effect is stronger than the indirect effect from increased competition with rivals.
Ruth: That’s an interesting trade-off—AI helping you go far versus AI making you compete harder against everyone else.
Warren: What about the second finding? They look at how changing the share of AI automated R andD tasks affects firm behavior, and it's non-monotonic.
Iris: Initially, when firms start using more AI in R andD, they tend to target more radical innovations right away because of that direct facilitation effect. But then beyond a certain threshold of human-AI complementarity, their focus shifts toward incremental innovations instead.
Ruth: So there’s a sweet spot for AI adoption where it helps with big ideas but then the firm realizes sticking with those big ideas isn't as advantageous anymore compared to smaller steps.
Warren: That makes sense in a market context—when the cost or risk of that radical jump outweighs the benefit, they settle for safer incremental gains.
Iris: And finally, in the extreme case where full automation happens, their model predicts that the optimal recombination distance collapses to zero, meaning fully AI-driven research would undermine what it’s trying to achieve.
Ruth: That’s a strong conclusion—that pushing AI all the way might actually hurt knowledge creation by eliminating the need for those long-distance bridges altogether.
Title and authors: Warren: So they are suggesting there's an optimal degree of AI integration, not just more is better.
Iris: The paper contributes by embedding this recombination problem into that quality ladder framework, which captures competitive dynamics of creative destruction in a way that models like Gans's don't do well.
Ruth: That’s a big piece of the puzzle—capturing those competitive dynamics explicitly within the structure itself.
Warren: And they also provide a way to model AI power as this composite index, which generates that non-monotonic relationship we talked about through those effects like the streetlight effect and stepping-on-toes effect.
Iris: Their implication is that policies should watch out for over-automation because it could lead to duplication of effort in scientific research.
Ruth: That points toward needing careful reconsideration of patent systems and the regulation of AI-assisted research because the duration of those monopolies matters so much here.
Warren: So, overall, they’re showing that AI isn't just a simple accelerator; it fundamentally reshapes the incentives around how we explore new scientific territory.
Iris: Before we wrap up, I want to mention their limitation: they treat AI as an exogenous input with a fixed price trajectory and don't model the strategic behavior of the AI providers themselves.
Ruth: That’s a fair point—the paper doesn't look at how companies *choose* to use different AI tools in terms of cost or capability development.
Warren: And they also suggest future work could explore how the structure of the knowledge graph itself influences innovation type, like how dense regions benefit most from those long-distance recombinations for knowledge expansion.
Iris: They also suggest empirical testing through things like patent citation networks and semantic similarity measures to see how the topology of that knowledge space evolves over time.
Ruth: And they even compared their model against scenarios with decreasing or fast-growing AI prices, which confirms that property rights and monopoly power still matter for growth rates.
Warren: So it’s a lot to chew on—the paper "Bridging Distant Ideas: the Impact of AI on R andD and Recombinant Innovation" gives us a model that shows how AI can push firms toward radical or incremental research based on their specific competitive environment.
The paper's summary: Ruth: So, to recap, this paper builds a model showing how AI messes with whether research teams choose to make small tweaks or try massive leaps in their ideas by looking at how those ideas are connected in a knowledge space.
Warren: Right. It’s about that tension between incremental work and radical innovation, and how AI acts as a force pushing firms in one direction or the other depending on the situation.
Ruth: The key finding is that higher AI productivity actually encourages firms to try those more distant recombinations—the big leaps—if the direct help from AI is stronger than how much it makes rivals compete against them.
Warren: That’s interesting because it suggests AI isn't just a generic tool; its impact depends on the competitive landscape. If you’re in a field where competition is fierce, AI might push you toward incremental steps instead of trying to go super far out.
Ruth: And they looked at how much R andD tasks are automated, and that relationship isn't smooth. It goes up when firms start, but then it starts going down once they hit a certain level of human-AI teamwork where the benefits level off, pushing them back toward incremental work.
Warren: That’s the non-monotonic part. So you get a boost for radical stuff initially, but if you automate too much, that benefit evaporates and firms settle for safer bets.
Ruth: And they have this really telling prediction about what happens when everything becomes fully automated. If AI takes over completely, their model suggests the best place to innovate is right here at zero distance—meaning no new connections are made, and knowledge creation stalls.
Warren: That’s a pretty stark warning about over-reliance on automation in scientific research. It implies that if we let AI do everything, we might actually stop generating new ideas because those distant connections become pointless.
Ruth: The authors contribute by putting this whole recombination problem into a quality ladder framework, which helps capture the competitive dynamics of creative destruction in a way other models don't quite manage.
Warren: And they also give us this composite index for AI power that generates that non-monotonic effect we just talked about, using those effects like the streetlight effect and stepping-on-toes effect.
Ruth: Their implication is that policymakers should watch out for over-automation because it can lead to duplication of effort in scientific research across the economy.
Warren: And they also point out that patent systems and how we regulate AI assisted research need rethinking because the duration of those monopolies matters so much in this model.
Ruth: So, what this means for us is that integrating AI into R andD isn't a simple scaling up thing; there’s an optimal point where using more AI actually hurts the creation of truly novel knowledge.
Warren: It sounds like a call to be strategic about where you deploy that technology within the research process.
Ruth: And before we move on to how this relates to the other papers we've discussed, remember that the model itself treats AI as an outside input with a fixed price path, so it doesn't account for firms strategically choosing what kind of AI they want to build or deploy.
The paper's improvements: Tom: So, we're looking at how the authors suggest ways to actually make this model better or more useful.
Ruth: They propose a few key tweaks that would give the whole system more realism, focusing on making the AI usage decision-making process less abstract.
Warren: What are these improvements? Are they changing what we measure or how we treat the AI input itself?
Ruth: They suggest optimizing research focus by dynamically adjusting how much AI a firm uses based on how complex the task is, showing that an increase in the fraction of tasks done by AI, let's call it alpha, boosts its power lambdaAI up to a certain point, and then it actually starts decreasing after that.
Warren: So they’re suggesting firms need a threshold before they stop chasing radical ideas and start settling for incremental ones.
Ruth: Exactly. Then there’s the idea of using the knowledge space structure better, where by treating it like a graph, you can identify dense regions where scientific understanding is already strong, and those areas benefit most from long-distance recombinations to open up entirely new research paths.
Warren: That makes sense for strategy—don't waste effort trying to bridge gaps where the foundation is already solid.
Ruth: They also suggest improving how they model creative destruction dynamics by explicitly accounting for how AI empowers competitors, which means it speeds up that rate of destruction and shortens the time those innovation monopolies last.
Warren: That gives us a clearer picture of the competitive pressure AI puts on the whole system, not just on individual firms.
Ruth: Then there's this idea to embed quality ladder management into research targets so firms can manage their optimal target distance, balancing the risk of risky long-distance recombinations against safer short-distance innovations.
Warren: So it’s about giving firms a better tool for risk management in their innovation pipeline.
Ruth: And they also look at developing a dynamic AI price forecasting mechanism that links the cost of AI to the growth of knowledge stock, because they assume the price of AI goes up as more data makes training and deploying it more expensive, which is mu t equals b armuAt.
Warren: That’s smart. It ties the cost structure directly to how much knowledge you're trying to accumulate.
Ruth: The authors flag a limitation here: they treat AI as an outside input with a fixed price trajectory and don't model the actual strategic behavior of the AI providers themselves, which is something they suggest future work should address.
Warren: So the model is great for showing incentives, but it doesn't explore how the companies that *make* the AI might be playing their own game.
Ruth: True. And they also suggest empirical testing through things like patent citation networks and semantic similarity measures to see how the topology of that knowledge space actually evolves in real-world data.
Warren: So, for anyone driving or cooking, think of this as a blueprint for understanding why a company decides to take a big swing in research versus playing it safe with smaller bets when they have powerful AI tools.
Conclusion: Tom: So, to wrap up this discussion on "Bridging Distant Ideas: The Impact of AI on R andD and Recombinant Innovation," we’re essentially talking about how AI changes the incentives firms have for making big versus small leaps in their research ideas.
Ruth: Right. The core message is that AI isn't just a tool; it fundamentally alters the trade-off between pursuing radical, far-reaching ideas and focusing on incremental improvements based on where they are in their knowledge space.
Warren: It shows that the impact depends entirely on the competitive environment, which is something I always like to think about when looking at any strategy.
Iris: From an econometric standpoint, it's interesting because the authors managed to identify the variation—the specific way AI adoption interacts with task complexity—that drives this non-monotonic relationship between innovation type and AI use.
Ruth: Exactly. They show that there’s an optimal level of AI integration where it helps with big ideas, but beyond that, firms shift focus toward incremental stuff because the complementarity effect just wears off.
Warren: And they also made a pretty strong prediction about what happens when you push the automation all the way—they find that optimal recombination distance collapses to zero, meaning full AI research might actually stop knowledge creation.
Iris: That’s a big warning. It suggests that if we automate everything, we lose the very mechanism of connecting disparate ideas that leads to true novelty.
Ruth: So, what this means for us is that policy needs to look at regulating the duration of innovation monopolies because AI shortens those times, and they should probably monitor risk levels to avoid duplication in research.
Warren: I agree on the regulation point. If innovation cycles get shorter because of faster AI competition, we need new ways to manage those patent systems.
Iris: And for future work, they suggest testing this stuff empirically using things like patent citation networks and semantic similarity measures to see if the knowledge space topology actually changes as predicted.
Ruth: So, in short, this paper gives us a framework for understanding how AI reshapes the research incentive structure based on these complex recombination dynamics.
Warren: It’s a solid piece of work because it connects economic incentives directly to the structural way ideas are connected.
Iris: It moves beyond just saying "AI is good" and shows precisely how its specific mechanisms—like those interaction effects—create the observed behavior in firms.
Ruth: We've covered a lot about this paper, but remember, it’s a model built on assumptions about how people behave in markets, so we always need to keep an eye on those assumptions.
Warren: That’s true. It’s a useful lens for seeing how AI plays out in the real world, even if the model isn't perfect.
Iris: We've talked about the mechanics and the results of "Bridging Distant Ideas: The Impact of AI on R andD and Recombinant Innovation."
Ruth: Indeed. Next time we look at arXiv papers, we’ll be diving into how these findings connect to other topics, like how teamwork affects investment in teams or how regulation handles inequality.
IMT School for Advanced Studies Lucca · University School for Advanced Studies of Pavia · TIK Centre University of Oslo
econ.TH
Submitted: 2026-04-02
Updated: 2026-11-13
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 92/100
The gist: We study how artificial intelligence (AI) affects firms’ incentives to pursue incremental versus radical knowledge recombinations This paper develops a model of recombinant innovation embedded in a…
Key concepts
- Recombinant Innovation
- This refers to creating new ideas by combining existing ones from different fields. The model treats this as navigating a 'knowledge space' where ideas are positioned based on how different they are cognitively. Firms must choose between combining very distant ideas or very close, familiar ones.
- Schumpeterian Quality-Ladder Framework
- This is a structural model that organizes innovation by placing ideas on a ladder based on their quality or distance in the knowledge space. It helps firms manage the trade-off between pursuing highly ambitious, distant research and safer, incremental improvements.
- AI's Dual Influence
- AI impacts R&D both positively and negatively. Positively, AI can make successful recombinations more likely by shrinking idea distances. Negatively, it empowers competitors by accelerating innovation cycles (creative destruction), which shortens the time firms have to maintain a monopoly.
- Non-monotonic Relationship
- This describes a relationship where the effect of an input changes direction. In this context, increasing AI adoption initially encourages radical research but then causes firms to pivot toward incremental research once a certain level of human-AI complementarity is reached.
Terminology
Summary
We study how artificial intelligence (AI) affects firms’ incentives to pursue incremental versus radical knowledge recombinations This paper develops a model of recombinant innovation embedded in a Schumpeterian quality-ladder framework to investigate the impact of AI on R&D and recombinant innovation. p. 0 Bridging Distant Ideas: the Impact of
AI on R&D and Recombinant
Innovation
Emanuele Bazzichi1
, Massimo Riccaboni1,2 and Fulvio Castellacci3
This version: 31st March 2026
Abstract
We study how artificial intelligence (AI) affects firms’ incentives to pursue incremental
versus radical knowledge recombinations. We develop a model of recombinant innovation embedded in a Schumpeterian quality-ladder framework, in which innovation
arises from recombining ideas across varying distances in a knowledge space.Where the paper says it
Model Framework
The study develops a model of recombinant innovation embedded in a Schumpeterian quality-ladder framework, where innovation arises from recombining ideas across varying distances in a knowledge space. This framework positions ideas in a graph structure where distances capture the cognitive dissimilarity between fields R&D firms choose how far to reach across this space, facing a fundamental trade-off between distant recombinations and close recombinations. p. 2 Ideas are located in a knowledge
space – a graph structure where distances capture the cognitive dissimilarity between fields. R&D firms choose how far to reach across this space when attempting to combine ideas,
facing a fundamental trade-off: distant recombinations are riskier but, conditional on success,
they lead to more radical and more profitable innovations, while close recombinations are
easier to develop but produce only incremental advances. Research is modeled using a taskbased production function in which a fraction of tasks can be performed by AI, while the
other tasks require human researchers. Innovation arrives as a Poisson process with an
arrival rate that decreases with recombinant distance, and each successful innovation grants
the innovator a temporary monopoly until displaced by the next one.Where the paper says it
Where the paper says it
AI's Dual Influence
AI enters this framework through four channels, two positive and two negative. The positive effects include AI increasing the probability that a given recombination attempt succeeds, effectively shrinking the distance between ideas and enabling more ambitious research strategies and indirectly expanding the frontier of future innovation opportunities by creating new intermediate connections in the knowledge space, analogous to diffusion effects documented in General Purpose Technologies literature. p. 2 On the
positive side, first, AI increases the probability that a given recombination attempt succeeds, effectively shrinking the distance between ideas and enabling more ambitious research
strategies. Second, by facilitating more distant recombinations, AI indirectly expands the
frontier of future innovation opportunities: a successful distant recombination creates new
intermediate connections in the knowledge space, opening shorter paths between previously
unrelated ideas and generating a wave of follow-up innovations – a mechanism analogous to
the diffusion effects documented in the General Purpose Technologies literature (Bresnahan
and Trajtenberg, 1995). On the negative side, first, AI also empowers competitors: by raising the aggregate arrival rate of innovations across the economy, it accelerates the rate of
creative destruction and it shortens the expected duration of innovation monopolies, thus
reducing the expected reward from risky long-distance recombinations. Second, excessive
reliance on AI erodes the complementarity between AI and human researchers, as AI-driven
research gravitates toward data-rich and well-explored domains (streetlight effect) and dif3Where the paper says it
Where the paper says it
Key Findings
The model obtains three main results regarding the impact of AI on research incentives. First, higher AI productivity encourages more distant recombinations if the direct facilitation effect is stronger than the indirect effect due to intensified competition from rivals. p. 0 We obtain three main results. First, higher AI
productivity encourages more distant recombinations, if the direct facilitation effect
is stronger than the indirect effect due to intensified competition from rivals. First, higher AI
productivity encourages more distant recombinations, if the direct facilitation effect
is stronger than the indirect effect due to intensified competition from rivals. Second, the effect of increasing the share of AI-automated R&D tasks is non-monotonic:
firms initially target more radical innovations, but beyond a threshold of human-AI
complementarity, they shift the focus toward incremental innovations. Second, the effect of increasing the share of AI-automated R&D tasks is non-monotonic:
firms initially target more radical innovations, but beyond a threshold of human-AI
complementarity, they shift the focus toward incremental innovations. Third, in the
limiting case of full automation, the model predicts that optimal recombination distance collapses to zero, suggesting that fully AI-driven research would undermine the
very knowledge creation that it seeks to accelerate.Where the paper says it
Where the paper says it
Where the paper says it
Theoretical Contributions and Implications
The paper contributes by embedding the recombination problem into a quality-ladder framework, capturing the competitive dynamics of creative destruction which is absent in models like Gans's. Furthermore, it provides a tractable representation of AI power as a composite index that endogenously generates the non-monotonic relationship between AI adoption and innovation novelty through the interaction of streetlight and stepping-on-toes effects. p. 3 Second, our formulation of AI power as a composite index that combines AI productivity,
the breadth of AI task coverage and the residual human contribution, provides a tractable
representation of AI’s role in research that endogenously generates the non-monotonic relationship between AI adoption and innovation novelty. This non-monotonicity arises from
the interaction between the streetlight and stepping-on-toes effects noted above, and the
resulting loss of human-AI complementarity. , 2024; Toner-Rodgers, 2024) – and that different researchers using similar AI tools may
converge on the same ideas, producing duplication of research effort – the stepping-on-toes
effect (Almeida et al. These recent findings suggest that the relationship between
AI adoption and research novelty may be more complex than simple productivity-enhancing
accounts would imply. In pharmaceutical
R&D, AI accelerates drug discovery but the effect is strongest for drugs with medium levels
of novelty and for firms with deep domain knowledge that can filter out false positives (Lou
and Wu, 2021; Tranchero, 2024). More broadly, there is growing concern that AI-assisted
research may exhibit a tendency to concentrate on well-explored knowledge domains where
data is abundant and AI systems perform best – the so-called streetlight effect (Tranchero
et al.Where the paper says it
Where the paper says it
Where the paper says it
Limitations and Future Directions
The model treats AI as an exogenous input with a given price trajectory, without modeling the strategic behavior of AI providers. Future extensions could study how the structure of the knowledge graph influences innovation type, such as how dense regions benefit from long-distance recombinations for knowledge expansion. p. 2 Second, by facilitating more distant recombinations, AI indirectly expands the
frontier of future innovation opportunities: a successful distant recombination creates new
intermediate connections in the knowledge space, opening shorter paths between previously
unrelated ideas and generating a wave of follow-up innovations – a mechanism analogous to
the diffusion effects documented in the General Purpose Technologies literature (Bresnahan
and Trajtenberg, 1995). On the negative side, first, AI also empowers competitors: by raising the aggregate arrival rate of innovations across the economy, it accelerates the rate of
creative destruction and it shortens the expected duration of innovation monopolies, thus
reducing the expected reward from risky long-distance recombinations. While the current analysis exploits only the distance dimension of this
structure, the graph representation provides a richer conceptual framework that motivates
the GPT-effect – whereby a successful distant recombination creates new shorter paths for
future innovations – and opens possible directions for future research, e. through empirical
operationalization using patent citation networks and semantic similarity measures.Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Optimize research focus by dynamically adjusting AI usage based on task complexity levels. The model shows that "an increase in a fraction of tasks performed by AI (alpha) increases AI power lambdaAI until a threshold level alpha <= alpha star, and it decreases it thereafter." This allows firms to shift from radical to incremental innovations when the complementarity erodes.
-
Integrate knowledge space structure into innovation strategy for better idea generation. By using a graph-based representation, the system can identify
dense regions of the network where scientific understanding is already wellestablished may benefit most from long-distance recombinations that open entirely new research trajectories (knowledge expansion).
-
Improve competitive response by modeling creative destruction dynamics explicitly. The model accounts for how AI
empowers competitors: by raising the aggregate arrival rate of innovations across the economy, it accelerates the rate of creative destruction and it shortens the expected duration of innovation monopolies.
This allows firms to anticipate and navigate competitive displacement more accurately. -
Implement quality-ladder management to balance risk and reward in research targets. By embedding ideas in a
quality-ladder framework,
firms can manage theiroptimal target distance
considering the trade-off between risky long-distance recombinations and safer short-distance innovations. -
Develop a dynamic AI price forecasting mechanism linked to knowledge stock growth. Since the model assumes
the price of AI as an increasing function of the knowledge stock in the economy, as it becomes more costly to train and deploy AI algorithms on an growing amount of data: mu t = barmuAt,
the system can anticipate future research costs based on predicted knowledge accumulation.
Abstract
We study how artificial intelligence (AI) affects firms' incentives to pursue incremental versus radical knowledge recombinations. We develop a model of recombinant innovation embedded in a Schumpeterian quality-ladder framework, in which innovation arises from recombining ideas across varying distances in a knowledge space. R&D consists of multiple tasks, a fraction of which can be performed by AI. AI facilitates access to distant knowledge domains, but at the same time it also increases the aggregate rate of creative destruction, shortening the monopoly duration that rewards radical innovations. We obtain three main results. First, the effect of increasing the share of AI-automated R&D tasks is non-monotonic under specific conditions: firms initially target more radical innovations, but beyond a threshold they shift the focus toward incremental innovations. Second, with falling optimal distance, we may have increasing expected log-growth of knowledge due to higher innovation frequency or a hump-shaped growth. Finally, we identify a sufficient condition under which the target distance chosen in the decentralized equilibrium is sub-optimal from a social welfare point of view.