|
on Computational Economics |
|
Issue of 2024‒01‒15
twenty-two papers chosen by |
| By: | Maochun Xu; Zixun Lan; Zheng Tao; Jiawei Du; Zongao Ye |
| Abstract: | Artificial Intelligence (AI) and Machine Learning (ML) are transforming the domain of Quantitative Trading (QT) through the deployment of advanced algorithms capable of sifting through extensive financial datasets to pinpoint lucrative investment openings. AI-driven models, particularly those employing ML techniques such as deep learning and reinforcement learning, have shown great prowess in predicting market trends and executing trades at a speed and accuracy that far surpass human capabilities. Its capacity to automate critical tasks, such as discerning market conditions and executing trading strategies, has been pivotal. However, persistent challenges exist in current QT methods, especially in effectively handling noisy and high-frequency financial data. Striking a balance between exploration and exploitation poses another challenge for AI-driven trading agents. To surmount these hurdles, our proposed solution, QTNet, introduces an adaptive trading model that autonomously formulates QT strategies through an intelligent trading agent. Incorporating deep reinforcement learning (DRL) with imitative learning methodologies, we bolster the proficiency of our model. To tackle the challenges posed by volatile financial datasets, we conceptualize the QT mechanism within the framework of a Partially Observable Markov Decision Process (POMDP). Moreover, by embedding imitative learning, the model can capitalize on traditional trading tactics, nurturing a balanced synergy between discovery and utilization. For a more realistic simulation, our trading agent undergoes training using minute-frequency data sourced from the live financial market. Experimental findings underscore the model's proficiency in extracting robust market features and its adaptability to diverse market conditions. |
| Date: | 2023–12 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2312.15730&r=cmp |
| By: | Diana Barro (Department of Economics, Ca' Foscari University of Venice); Luca Barzanti (Department of Mathematics, University of Bologna); Marco Corazza (Department of Economics, Ca' Foscari University of Venice); Martina Nardon (Department of Economics, Ca' Foscari University of Venice) |
| Abstract: | In fundraising management, the assessment of the expected gift is a key point. The availability of accurate estimates of the number of donations, their amounts, and the gift probability is relevant in order to evaluate the results of a fundraising campaign. The accuracy of the expected gift estimation depends on the appropriate use of the information about Donors. In this contribution, we propose a non-parametric methodology for the prediction of Donors' behavior based on Artificial Neural Networks. In particular, Multi-Layer Perceptron is applied. In the numerical experiments, the expected gift is then estimated based on a simulated dataset of Donors' individual characteristics and information on donations history. |
| Keywords: | Fundraising Management, Donor's Profile, Gift Expectation, Artificial Neural Networks |
| JEL: | C45 D64 |
| Date: | 2023 |
| URL: | http://d.repec.org/n?u=RePEc:ven:wpaper:2023:33&r=cmp |
| By: | Ramaharo, Franck Maminirina (Ministry of Economy and Finance (Ministère de l'Economie et des Finances)); Rasolofomanana, Gerzhino H (Ministry of Economy and Finances) |
| Abstract: | We investigate the predictive power of different machine learning algorithms to nowcast Madagascar's gross domestic product (GDP). We trained popular regression models, including linear regularized regression (Ridge, Lasso, Elastic-net), dimensionality reduction model (principal component regression), k-nearest neighbors algorithm (k-NN regression), support vector regression (linear SVR), and tree-based ensemble models (Random forest and XGBoost regressions), on 10 Malagasy quarterly macroeconomic leading indicators over the period 2007Q1-2022Q4, and we used simple econometric models as a benchmark. We measured the nowcast accuracy of each model by calculating the root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE). Our findings reveal that the Ensemble Model, formed by aggregating individual predictions, consistently outperforms traditional econometric models. We conclude that machine learning models can deliver more accurate and timely nowcasts of Malagasy economic performance and provide policymakers with additional guidance for data-driven decision making. |
| Date: | 2023–12–22 |
| URL: | http://d.repec.org/n?u=RePEc:osf:africa:vpuac&r=cmp |
| By: | Richiardi, Matteo; Bronka, Patryk; van de Ven, Justin |
| Abstract: | In this chapter we focus on the commonalities and differences between agent-based and dynamic microsimulation analytical approaches. Starting from a shared history, we discuss how the two literatures quickly diverged. Discussion concludes with evidence of some recent convergence between agent-based and dynamic microsimulation methods, and emerging opportunities for mutual reinforcement of the two methodologies. |
| Date: | 2023–12–01 |
| URL: | http://d.repec.org/n?u=RePEc:ese:cempwp:cempa8-23&r=cmp |
| By: | Jacques Bughin |
| Abstract: | Companies must master AI—and all techniques, whether it is (un)supervised learning or reinforcement learning as is set to revolutionise predictive powers and maximise chances of success in sports and other fields. |
| Keywords: | ChatGPT, AI |
| Date: | 2023–12 |
| URL: | http://d.repec.org/n?u=RePEc:ict:wpaper:2013/365936&r=cmp |
| By: | Luca Gaegauf (University of Zurich); Simon Scheidegger (University of Lausanne); Fabio Trojani (University of Geneva; University of Turin; Swiss Finance Institute) |
| Abstract: | We introduce a comprehensive computational framework for solving dynamic portfolio choice problems with many risky assets, transaction costs, and borrowing and short-selling constraints. Our approach leverages the synergy between Gaussian process regression and Bayesian active learning to efficiently approximate value and policy functions with a novel, formal way of characterizing the irregularly-shaped no-trade region; we then embed this into a discrete-time dynamic programming algorithm. This combination allows us to study dynamic portfolio choice problems with more risky assets than was previously possible. Our results indicate that giving the agent access to more assets may alleviate some illiquidity resulting from the presence of transaction costs. |
| Keywords: | Machine learning, computational finance, computational economics, Gaussian process regression, dynamic portfolio optimization, transaction costs, liquidity premia |
| JEL: | C61 C63 C68 E21 |
| Date: | 2023–11 |
| URL: | http://d.repec.org/n?u=RePEc:chf:rpseri:rp23114&r=cmp |
| By: | HRADEC Jiri (European Commission - JRC); OSTLAENDER Nicole; BERNINI Alba |
| Abstract: | The research investigates how large language models (LLMs) emerge as reservoirs of a vast array of human experiences, behaviours, and emotions. Building upon prior work of the JRC on synthetic populations , it presents a complete step-by-step guide on how to use LLMs to create highly realistic modelling scenarios and complex societies of autonomous emotional AI agents. This technique is aligned with agent-based modelling (ABM) and facilitates quantitative evaluation. The report describes how the agents were instantiated using LLMs, enriched with personality traits using the ABC-EBDI model, equipped with short- and long-term memory, and access to detailed knowledge of their environment. This setting of embodied reasoning significantly improved the agents' problem-solving capabilities and when subjected to various scenarios, the LLM-driven agents exhibited behaviours mirroring human-like reasoning and emotions, inter-agent patterns and realistic conversations, including elements that mirrored critical thinking. These LLM-driven agents can serve as believable proxies for human behaviour in simulated environments presenting vast implications for future research and policy applications, including studying impacts of different policy scenarios. This bears the opportunity to combine the narrative-based world of foresight scenarios with the advantages of quantitative modelling |
| Date: | 2023–10 |
| URL: | http://d.repec.org/n?u=RePEc:ipt:iptwpa:jrc135070&r=cmp |
| By: | Wladimir Ostrovsky |
| Abstract: | This paper explores the utility of agent-based simulations in realistically modelling market structures and sheds light on the nuances of optimal dealer strategies. It underscores the contrast between conclusions drawn from probabilistic modelling and agent-based simulations, but also highlights the importance of employing a realistic test bed to analyse intricate dynamics. This is achieved by extending the agent-based model for auction markets by \cite{Chiarella.2008} to include liquidity providers. By constantly and passively quoting, the dealers influence their own wealth but also have ramifications on the market as a whole and the other participating agents. Through synthetic market simulations, the optimal behaviour of different dealer strategies and their consequences on market dynamics are examined. The analysis reveals that dealers exhibiting greater risk aversion tend to yield better performance outcomes. The choice of quote sizes by dealers is strategy-dependent: one strategy demonstrates enhanced performance with larger quote sizes, whereas the other strategy show a better results with smaller ones. Increasing quote size shows positive influence on the market in terms of volatility and kurtosis with both dealer strategies. However, the impact stemming from larger risk aversion is mixed. While one of the dealer strategies shows no discernible effect, the other strategy results in mixed outcomes, encompassing both positive and negative effects. |
| Date: | 2023–12 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2312.05943&r=cmp |
| By: | Neil Kichler; Sher Afghan; Uwe Naumann |
| Abstract: | The increasing use of stochastic models for describing complex phenomena warrants surrogate models that capture the reference model characteristics at a fraction of the computational cost, foregoing potentially expensive Monte Carlo simulation. The predominant approach of fitting a large neural network and then pruning it to a reduced size has commonly neglected shortcomings. The produced surrogate models often will not capture the sensitivities and uncertainties inherent in the original model. In particular, (higher-order) derivative information of such surrogates could differ drastically. Given a large enough network, we expect this derivative information to match. However, the pruned model will almost certainly not share this behavior. In this paper, we propose to find surrogate models by using sensitivity information throughout the learning and pruning process. We build on work using Interval Adjoint Significance Analysis for pruning and combine it with the recent advancements in Sobolev Training to accurately model the original sensitivity information in the pruned neural network based surrogate model. We experimentally underpin the method on an example of pricing a multidimensional Basket option modelled through a stochastic differential equation with Brownian motion. The proposed method is, however, not limited to the domain of quantitative finance, which was chosen as a case study for intuitive interpretations of the sensitivities. It serves as a foundation for building further surrogate modelling techniques considering sensitivity information. |
| Date: | 2023–12 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2312.03510&r=cmp |
| By: | Stefano Bianchini; Moritz M\"uller; Pierre Pelletier |
| Abstract: | New technologies have the power to revolutionize science. It has happened in the past and is happening again with the emergence of new computational tools, such as Artificial Intelligence (AI) and Machine Learning (ML). Despite the documented impact of these technologies, there remains a significant gap in understanding the process of their adoption within the scientific community. In this paper, we draw on theories of scientific and technical human capital (STHC) to study the integration of AI in scientific research, focusing on the human capital of scientists and the external resources available within their network of collaborators and institutions. We validate our hypotheses on a large sample of publications from OpenAlex, covering all sciences from 1980 to 2020. We find that the diffusion of AI is strongly driven by social mechanisms that organize the deployment and creation of human capital that complements the technology. Our results suggest that AI is pioneered by domain scientists with a `taste for exploration' and who are embedded in a network rich of computer scientists, experienced AI scientists and early-career researchers; they also come from institutions with high citation impact and a relatively strong publication history on AI. The pattern is similar across scientific disciplines, the exception being access to high-performance computing (HPC), which is important in chemistry and the medical sciences but less so in other fields. Once AI is integrated into research, most adoption factors continue to influence its subsequent reuse. Implications for the organization and management of science in the evolving era of AI-driven discovery are discussed. |
| Date: | 2023–12 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2312.09843&r=cmp |
| By: | Yan Leng; Yuan Yuan |
| Abstract: | The advances of Large Language Models (LLMs) are expanding their utility in both academic research and practical applications. Recent social science research has explored the use of these "black-box" LLM agents for simulating complex social systems and potentially substituting human subjects in experiments. Our study delves into this emerging domain, investigating the extent to which LLMs exhibit key social interaction principles, such as social learning, social preference, and cooperative behavior, in their interactions with humans and other agents. We develop a novel framework for our study, wherein classical laboratory experiments involving human subjects are adapted to use LLM agents. This approach involves step-by-step reasoning that mirrors human cognitive processes and zero-shot learning to assess the innate preferences of LLMs. Our analysis of LLM agents' behavior includes both the primary effects and an in-depth examination of the underlying mechanisms. Focusing on GPT-4, the state-of-the-art LLM, our analyses suggest that LLM agents appear to exhibit a range of human-like social behaviors such as distributional and reciprocity preferences, responsiveness to group identity cues, engagement in indirect reciprocity, and social learning capabilities. However, our analysis also reveals notable differences: LLMs demonstrate a pronounced fairness preference, weaker positive reciprocity, and a more calculating approach in social learning compared to humans. These insights indicate that while LLMs hold great promise for applications in social science research, such as in laboratory experiments and agent-based modeling, the subtle behavioral differences between LLM agents and humans warrant further investigation. Careful examination and development of protocols in evaluating the social behaviors of LLMs are necessary before directly applying these models to emulate human behavior. |
| Date: | 2023–12 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2312.15198&r=cmp |
| By: | Marcin Rzeszutek (Faculty of Psychology, University of Warsaw, Poland); Jorgen Vitting Andersen (CNRS, Centre d'Economie de la Sorbonne); Adam Szyszka (Warsaw School of Economics, Collegium of World Economy, Warsaw, Poland); Szymon Talaga (The Robert Zajonc Institute for Social Studies, University of Warsaw, Poland) |
| Abstract: | This study aims at connecting the behavioral corporate finance (micro level) perspective and complexity theory along with agent-based modelling in order to analyze the impact of selected behavioral managerial factors on aggregated data related to the financial market stability (macro level). Specifically, we want to explore whether subjective well-being (SWB) of corporate managers (CEOs) impacted their business decisions during the Covid-19 pandemic, and how it may be related to volatility of stock prices and the issue of financial stability during this critical period. Our study is based on a survey of 255 managers of companies listed on the Warsaw Stock Exchange in Poland over the period . Using the results of this survey, we build an agent-based model (ABM) calibrated for the specific case of Poland to investigate how decision making of CEOs, stemming from their SWB, influence the stock prices and selected financial market dynamics indicators. The results of our study indicate that the excess volatility of stock prices may be a function of changes of SWB of managers, which in turn could lead to some crashes on the macro level with respect to financial stability |
| Keywords: | subjective well-being; CEO; Covid-19; agent-based model |
| Date: | 2023–12 |
| URL: | http://d.repec.org/n?u=RePEc:mse:cesdoc:23017&r=cmp |
| By: | Xiangyu Cui; Xun Li; Yun Shi; Si Zhao |
| Abstract: | This paper studies a discrete-time mean-variance model based on reinforcement learning. Compared with its continuous-time counterpart in \cite{zhou2020mv}, the discrete-time model makes more general assumptions about the asset's return distribution. Using entropy to measure the cost of exploration, we derive the optimal investment strategy, whose density function is also Gaussian type. Additionally, we design the corresponding reinforcement learning algorithm. Both simulation experiments and empirical analysis indicate that our discrete-time model exhibits better applicability when analyzing real-world data than the continuous-time model. |
| Date: | 2023–12 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2312.15385&r=cmp |
| By: | Jacques Bughin |
| Abstract: | Generative artificial intelligence is invading the corporate suite and boardroom, surrounding the pros and cons of its use in the enterprise. Thefirst fundamental question is whether generative AI is truly a game changer, as evidenced by basic metrics such as productivity gains. We find thatit is, but that the benefits vary considerably from case to case, suggesting that managers need to do their homework to define their most favourableposition with respect to generative AI. |
| Keywords: | Generative artificial intelligence, AI |
| Date: | 2023–12 |
| URL: | http://d.repec.org/n?u=RePEc:ict:wpaper:2013/365939&r=cmp |
| By: | Soheila Khajoui; Saeid Dehyadegari; Sayyed Abdolmajid Jalaee |
| Abstract: | The present study aimed to forecast the exports of a select group of Organization for Economic Co-operation and Development (OECD) countries and Iran using the neural networks. The data concerning the exports of the above countries from 1970 to 2019 were collected. The collected data were implemented to forecast the exports of the investigated countries for 2021 to 2025. The analysis was performed using the Multi-Layer-Perceptron (MLP) neural network in Python. Out of the total number, 75 percent were used as training data, and 25 percent were used as the test data. The findings of the study were evaluated with 99% accuracy, which indicated the reliability of the output of the network. The Results show that Covid-19 has affected exports over time. However, long-term export contracts are less affected by tensions and crises, due to the effect of exports on economic growth, per capita income and it is better for economic policies of countries to use long-term export contracts. |
| Date: | 2023–12 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2312.15535&r=cmp |
| By: | Jingwei Ni (ETH Zurich); Julia Bingler (University of Oxford); Chiara Colesanti Senni (ETH Zürich; University of Zurich); Mathias Kraus (University of Erlangen); Glen Gostlow (University of Zurich); Tobias Schimanski (University of Zurich); Dominik Stammbach (ETH Zurich); Saeid Vaghefi (University of Zurich); Qian Wang (University of Zurich); Nicolas Webersinke (Friedrich-Alexander-Universität Erlangen-Nürnberg); Tobias Wekhof (ETH Zürich); Tingyu Yu (University of Zurich); Markus Leippold (University of Zurich; Swiss Finance Institute) |
| Abstract: | This paper introduces a novel approach to enhance Large Language Models (LLMs) with expert knowledge to automate the analysis of corporate sustainability reports by benchmarking them against the Task Force for Climate-Related Financial Disclosures (TCFD) recommendations. Corporate sustainability reports are crucial in assessing organizations' environmental and social risks and impacts. However, analyzing these reports' vast amounts of information makes human analysis often too costly. As a result, only a few entities worldwide have the resources to analyze these reports, which could lead to a lack of transparency. While AI-powered tools can automatically analyze the data, they are prone to inaccuracies as they lack domain-specific expertise. This paper introduces a novel approach to enhance LLMs with expert knowledge to automate the analysis of corporate sustainability reports. We christen our tool \textsc{chatReport}, and apply it in a first use case to assess corporate climate risk disclosures following the TCFD recommendations. ChatReport results from collaborating with experts in climate science, finance, economic policy, and computer science, demonstrating how domain experts can be involved in developing AI tools. We make our prompt templates, generated data, and scores available to the public to encourage transparency. |
| Keywords: | Task Force for Climate-Related Financial Disclosures, Sustainability Report, Large Language Model, ChatGPT |
| Date: | 2023–11 |
| URL: | http://d.repec.org/n?u=RePEc:chf:rpseri:rp23111&r=cmp |
| By: | Zhiyang Jia; Bodil M. Larsen; Bård Lian; Runa Nesbakken; Odd E. Nygård; Thor O. Thoresen; Trine E. Vattø (Statistics Norway) |
| Abstract: | Microsimulation models of the LOTTE system are key tools for tax policy-making in Norway and are extensively used in the budget process. The aim of this paper is to give an overview of the different modules in the LOTTE family – a non-behavioral tax-benefit model for personal income tax (LOTTESkatt), a labor supply model (LOTTE-Arbeid), and a model for distributional effects of commodity taxation (LOTTE-Konsum). In addition to providing descriptions of the designs of the three microsimulation models, we give examples of how the models are used in practical and academic work. |
| Keywords: | microsimulation; tax-benefit model; labor supply; commodity taxation |
| JEL: | C63 H24 H31 |
| Date: | 2023–11 |
| URL: | http://d.repec.org/n?u=RePEc:ssb:dispap:1009&r=cmp |
| By: | Jean Lee; Hoyoul Luis Youn; Josiah Poon; Soyeon Caren Han |
| Abstract: | There has been growing interest in applying NLP techniques in the financial domain, however, resources are extremely limited. This paper introduces StockEmotions, a new dataset for detecting emotions in the stock market that consists of 10, 000 English comments collected from StockTwits, a financial social media platform. Inspired by behavioral finance, it proposes 12 fine-grained emotion classes that span the roller coaster of investor emotion. Unlike existing financial sentiment datasets, StockEmotions presents granular features such as investor sentiment classes, fine-grained emotions, emojis, and time series data. To demonstrate the usability of the dataset, we perform a dataset analysis and conduct experimental downstream tasks. For financial sentiment/emotion classification tasks, DistilBERT outperforms other baselines, and for multivariate time series forecasting, a Temporal Attention LSTM model combining price index, text, and emotion features achieves the best performance than using a single feature. |
| Date: | 2023–01 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2301.09279&r=cmp |
| By: | Harashima, Taiji |
| Abstract: | In this paper, I simulate how an economy grows endogenously and reaches a balanced growth path supposing that households behave under the MDC (maximum degree of comfortability)-based procedure, where MDC indicates the state at which a household feels most comfortable with its combination of income and assets. Although it is not easy to numerically simulate the path to a steady state in dynamic economic growth models in which households behave generating rational expectations, it is easy if households are supposed to behave under the MDC-based procedure to reach a steady state. The simulation results indicate that an economy can indeed grow endogenously as predicted theoretically, although some small scale effects exist. If uncompensated knowledge spillovers are restrained, however, large scale effects are generated. A lower degree of risk aversion increases the growth rate. In addition, economies converge if productivities are identical, but they diverge if they are not. |
| Keywords: | Convergence; Endogenous growth; Scale effects; Simulation; Uncompensated knowledge spillovers |
| JEL: | E17 E60 O11 O30 O40 |
| Date: | 2023–12–08 |
| URL: | http://d.repec.org/n?u=RePEc:pra:mprapa:119391&r=cmp |
| By: | Anil R. Doshi; Oliver P. Hauser |
| Abstract: | Creativity is core to being human. Generative artificial intelligence (GenAI) holds promise for humans to be more creative by offering new ideas, or less creative by anchoring on GenAI ideas. We study the causal impact of GenAI ideas on the production of an unstructured creative output in an online experimental study where some writers could obtain ideas for a story from a GenAI platform. We find that access to GenAI ideas causes stories to be evaluated as more creative, better written and more enjoyable, especially among less creative writers. However, objective measures of story similarity within each condition reveal that GenAI-enabled stories are more similar to each other than stories by humans alone. These results point to an increase in individual creativity, but at the same time there is a risk of losing collective novelty: this dynamic resembles a social dilemma where individual writers are better off using GenAI to improve their own writing, but collectively a narrower scope of novel content may be produced with GenAI. Our results have implications for researchers, policy-makers and practitioners interested in bolstering creativity, but point to potential downstream consequences from over-reliance. |
| Date: | 2023–12 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2312.00506&r=cmp |
| By: | von der Heyde, Leah (LMU Munich); Haensch, Anna-Carolina; Wenz, Alexander (University of Mannheim) |
| Abstract: | The recent development of large language models (LLMs) has spurred discussions about whether LLM-generated “synthetic samples” could complement or replace traditional surveys, considering their training data potentially reflects attitudes and behaviors prevalent in the population. A number of mostly US-based studies have prompted LLMs to mimic survey respondents, finding that the responses closely match the survey data. However, several contextual factors related to the relationship between the respective target population and LLM training data might affect the generalizability of such findings. In this study, we investigate the extent to which LLMs can estimate public opinion in Germany, using the example of vote choice as outcome of interest. To generate a synthetic sample of eligible voters in Germany, we create personas matching the individual characteristics of the 2017 German Longitudinal Election Study respondents. Prompting GPT-3 with each persona, we ask the LLM to predict each respondents’ vote choice in the 2017 German federal elections and compare these predictions to the survey-based estimates on the aggregate and subgroup levels. We find that GPT-3 does not predict citizens’ vote choice accurately, exhibiting a bias towards the Green and Left parties, and making better predictions for more “typical” voter subgroups. While the language model is able to capture broad-brush tendencies tied to partisanship, it tends to miss out on the multifaceted factors that sway individual voter choices. Furthermore, our results suggest that GPT-3 might not be reliable for estimating nuanced, subgroup-specific political attitudes. By examining the prediction of voting behavior using LLMs in a new context, our study contributes to the growing body of research about the conditions under which LLMs can be leveraged for studying public opinion. The findings point to disparities in opinion representation in LLMs and underscore the limitation of applying them for public opinion estimation without accounting for the biases in their training data. |
| Date: | 2023–12–15 |
| URL: | http://d.repec.org/n?u=RePEc:osf:socarx:8je9g&r=cmp |
| By: | George Gui; Olivier Toubia |
| Abstract: | Large Language Models (LLMs) have demonstrated impressive potential to simulate human behavior. Using a causal inference framework, we empirically and theoretically analyze the challenges of conducting LLM-simulated experiments, and explore potential solutions. In the context of demand estimation, we show that variations in the treatment included in the prompt (e.g., price of focal product) can cause variations in unspecified confounding factors (e.g., price of competitors, historical prices, outside temperature), introducing endogeneity and yielding implausibly flat demand curves. We propose a theoretical framework suggesting this endogeneity issue generalizes to other contexts and won't be fully resolved by merely improving the training data. Unlike real experiments where researchers assign pre-existing units across conditions, LLMs simulate units based on the entire prompt, which includes the description of the treatment. Therefore, due to associations in the training data, the characteristics of individuals and environments simulated by the LLM can be affected by the treatment assignment. We explore two potential solutions. The first specifies all contextual variables that affect both treatment and outcome, which we demonstrate to be challenging for a general-purpose LLM. The second explicitly specifies the source of treatment variation in the prompt given to the LLM (e.g., by informing the LLM that the store is running an experiment). While this approach only allows the estimation of a conditional average treatment effect that depends on the specific experimental design, it provides valuable directional results for exploratory analysis. |
| Date: | 2023–12 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2312.15524&r=cmp |