|
on Big Data |
| By: | Erik Brynjolfsson; Danielle Li; Lindsey Raymond |
| Abstract: | We study the staggered introduction of a generative AI-based conversational assistant using data from 5, 000 customer support agents. Access to the tool increases productivity, as measured by issues resolved per hour, by 14 percent on average, with the greatest impact on novice and low-skilled workers, and minimal impact on experienced and highly skilled workers. We provide suggestive evidence that the AI model disseminates the potentially tacit knowledge of more able workers and helps newer workers move down the experience curve. In addition, we show that AI assistance improves customer sentiment, reduces requests for managerial intervention, and improves employee retention. |
| Date: | 2023–04 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2304.11771&r=big |
| By: | Kreitmeir, David; Raschky, Paul Anton (Monash University) |
| Abstract: | We analyse the effects of the ban of ChatGPT, a generative pre-trained transformer chatbot, on individual productivity. We first compile data on the hourly coding output of over 8, 000 professional GitHub users in Italy and other European countries to analyse the impact of the ban on individual productivity. Combining the high-frequency data with the sudden announcement of the ban in a difference-in-differences framework, we find that the output of Italian developers decreased by around 50\% in the first two business days after the ban and recovered after that. Applying a synthetic control approach to daily Google search and Tor usage data shows that the ban led to a significant increase in the use of censorship bypassing tools. Our findings show that users swiftly implement strategies to bypass Internet restrictions but this adaptation activity creates short-term disruptions and hampers productivity. |
| Date: | 2023–04–18 |
| URL: | http://d.repec.org/n?u=RePEc:osf:socarx:v3cgs&r=big |
| By: | John J. Horton |
| Abstract: | Newly-developed large language models (LLM)—because of how they are trained and designed—are implicit computational models of humans—a homo silicus. LLMs can be used like economists use homo economicus: they can be given endowments, information, preferences, and so on, and then their behavior can be explored in scenarios via simulation. Experiments using this approach, derived from Charness and Rabin (2002), Kahneman, Knetsch and Thaler (1986), and Samuelson and Zeckhauser (1988) show qualitatively similar results to the original, but it is also easy to try variations for fresh insights. LLMs could allow researchers to pilot studies via simulation first, searching for novel social science insights to test in the real world. |
| JEL: | D0 |
| Date: | 2023–04 |
| URL: | http://d.repec.org/n?u=RePEc:nbr:nberwo:31122&r=big |
| By: | Kasy, Maximilian |
| Abstract: | This chapter discusses the regulation of artificial intelligence (AI) from the vantage point of political economy, based on the following premises: (i) AI systems maximize a single, measurable objective. (ii) In society, different individuals have different objectives. AI systems generate winners and losers. (iii) Society-level assessments of AI require trading off individual gains and losses. (iv) AI requires democratic control of algorithms, data, and computational infrastructure, to align algorithm objectives and social welfare. I address several debates regarding the ethics and social impact of AI, including (i) fairness, discrimination, and inequality, (ii) privacy, data property rights, and data governance, (iii) value alignment and the impending robot apocalypse, (iv) explainability and accountability for automated decision-making, and (v) automation and the impact of AI on the labor market and on wage inequality. (Stone Center on Socio-Economic Inequality Working Paper) |
| Date: | 2023–04–19 |
| URL: | http://d.repec.org/n?u=RePEc:osf:socarx:x7pcy&r=big |
| By: | Maximilian Andres |
| Abstract: | So far, the theory of equilibrium selection in the infinitely repeated prisoner's dilemma is insensitive to communication possibilities. To address this issue, we incorporate the assumption that communication reduces -- but does not entirely eliminate -- an agent's uncertainty that the other agent follows a cooperative strategy into the theory. Because of this, agents still worry about the payoff from cooperating when the other one defects, i.e. the sucker's payoff S, and, games with communication are more conducive to cooperation than games without communication. This theory is supported by data from laboratory experiments, and by machine learning based evaluation of the communication content. |
| Date: | 2023–04 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2304.12297&r=big |
| By: | Erik Brynjolfsson; Danielle Li; Lindsey R. Raymond |
| Abstract: | We study the staggered introduction of a generative AI-based conversational assistant using data from 5, 179 customer support agents. Access to the tool increases productivity, as measured by issues resolved per hour, by 14 percent on average, with the greatest impact on novice and low-skilled workers, and minimal impact on experienced and highly skilled workers. We provide suggestive evidence that the AI model disseminates the potentially tacit knowledge of more able workers and helps newer workers move down the experience curve. In addition, we show that AI assistance improves customer sentiment, reduces requests for managerial intervention, and improves employee retention. |
| JEL: | D8 J24 M15 M51 O33 |
| Date: | 2023–04 |
| URL: | http://d.repec.org/n?u=RePEc:nbr:nberwo:31161&r=big |
| By: | Leonard, Bryan; Gigliotti, Laura; Middleton, Arthur; Kroetz, Kailin (Resources for the Future) |
| Abstract: | Cost-effective conservation program design to support seasonal migratory species is urgently needed, but to-date has received little attention by economists. Conserving migratory corridors is a complicated design problem because of the large spatial scales over which migratory species can travel and the weakest-link characteristic of the problem. If one section or area of a potential migratory corridor is unable to support species movement, the migration through that route will not be successful. We develop and apply an integer-programming modeling approach that leverages innovative new data products to propose a cost-effective, landscape-scale conservation planning approach. We apply our approach to the Cody elk herd range within the Greater Yellowstone Ecosystem (GYE), leveraging satellite data on crop type and density over time and GPS collar data on elk migrations. We provide empirical evidence that using new satellite data products can avoid unconnected corridors and increase the cost effectiveness of corridor construction. In the Cody context, we estimate that achieving the conservation outcome associated with using satellite data on both costs and benefits would cost close to twice as much when using satellite benefit data but only limited cost data and about three times as much when using satellite cost data but only limited benefit data. Empirical work across additional herds is needed to provide additional insights into characteristics of contexts under which we expect gains from satellite and/or GPS collar data. |
| Date: | 2022–10–19 |
| URL: | http://d.repec.org/n?u=RePEc:rff:dpaper:dp-22-21&r=big |
| By: | Zhou, Janet; Azelton, Krystal; Nassar, Isabelle-Yara; Borowitz, Mariel |
| Abstract: | In 2014, Nigeria halted transmission of wild polio virus for the first time in its history. A critical enabling component in this historic achievement was the use of satellite data to produce more accurate maps and population estimates used in planning and implementing vaccination campaigns. This paper employs a value of information approach to estimate the net socioeconomic benefits associated with this use of satellite data. We calculate the increase in the likelihood of halting transmission of polio associated with the use of the satellite-based information compared to traditional data sources, and we consider the benefits associated with savings to the healthcare system as well as health benefits. Using a conservative approach focused on just one year of benefits, we estimate net socioeconomic benefits of between $46.0 million and $153.9 million. In addition to these quantified benefits, we also recognize qualitative benefits associated with improving human health, reaching marginalized communities, and building capacity among local populations. We also explore the substantial benefits associated with follow-on projects that have made use of the satellite-based data products and methodologies originally developed for the Nigeria polio eradication effort. |
| Date: | 2022–10–19 |
| URL: | http://d.repec.org/n?u=RePEc:rff:dpaper:dp-22-20&r=big |
| By: | Andrés Alonso-Robisco (Banco de España); José Manuel Carbó (Banco de España); José Manuel Marqués (Banco de España) |
| Abstract: | Preventing the materialization of climate change is one of the main challenges of our time. The involvement of the financial sector is a fundamental pillar in this task, which has led to the emergence of a new field in the literature, climate finance. In turn, the use of Machine Learning (ML) as a tool to analyze climate finance is on the rise, due to the need to use big data to collect new climate-related information and model complex non-linear relationships. Considering the proliferation of articles in this field, and the potential for the use of ML, we propose a review of the academic literature to assess how ML is enabling climate finance to scale up. The main contribution of this paper is to provide a structure of application domains in a highly fragmented research field, aiming to spur further innovative work from ML experts. To pursue this objective, first we perform a systematic search of three scientific databases to assemble a corpus of relevant studies. Using topic modeling (Latent Dirichlet Allocation) we uncover representative thematic clusters. This allows us to statistically identify seven granular areas where ML is playing a significant role in climate finance literature: natural hazards, biodiversity, agricultural risk, carbon markets, energy economics, ESG factors & investing, and climate data. Second, we perform an analysis highlighting publication trends; and thirdly, we show a breakdown of ML methods applied by research area. |
| Keywords: | climate finance, machine learning, literature review, Latent Dirichlet Allocation |
| JEL: | L93 R4 R11 |
| Date: | 2023–02 |
| URL: | http://d.repec.org/n?u=RePEc:bde:wpaper:2310&r=big |
| By: | Vitaly Meursault; Daniel Moulton; Larry Santucci; Nathan Schor |
| Abstract: | Modeling advances create credit scores that predict default better overall, but raise concerns about their effect on protected groups. Focusing on low- and moderate-income (LMI) areas, we use an approach from the Fairness in Machine Learning literature — fairness constraints via group-specific prediction thresholds — and show that gaps in true positive rates (% of non-defaulters identified by the model as such) can be significantly reduced if separate thresholds can be chosen for non-LMI and LMI tracts. However, the reduction isn’t free as more defaulters are classified as good risks, potentially affecting both consumers’ welfare and lenders’ profits. The trade-offs become more favorable if the introduction of fairness constraints is paired with the introduction of more sophisticated models, suggesting a way forward. Overall, our results highlight the potential benefits of explicitly considering sensitive attributes in the design of loan approval policies and the potential benefits of output-based approaches to fairness in lending. |
| Keywords: | Credit Scores; Group Disparities; Machine Learning; Fairness |
| JEL: | G51 C38 C53 |
| Date: | 2022–11–21 |
| URL: | http://d.repec.org/n?u=RePEc:fip:fedpwp:95158&r=big |
| By: | Simon, Frederik; Weibels, Sebastian; Zimmermann, Tom |
| Abstract: | We directly optimize portfolio weights as a function of firm characteristics via deep neural networks by generalizing the parametric portfolio policy framework. Our results show that network-based portfolio policies result in an increase of investor utility of between 30 and 100 percent over a comparable linear portfolio policy, depending on whether portfolio restrictions on individual stock weights, short-selling or transaction costs are imposed, and depending on an investor's utility function. We provide extensive model interpretation and show that network-based policies better capture the non-linear relationship between investor utility and firm characteristics. Improvements can be traced to both variable interactions and non-linearity in functional form. Both the linear and the network-based approach agree on the same dominant predictors, namely past return-based firm characteristics. |
| Keywords: | Portfolio Choice, Machine Learning, Expected Utility |
| JEL: | G11 G12 C58 C45 |
| Date: | 2023 |
| URL: | http://d.repec.org/n?u=RePEc:zbw:cfrwps:2301&r=big |
| By: | Kyungsub Lee |
| Abstract: | This study examines the use of a recurrent neural network for estimating the parameters of a Hawkes model based on high-frequency financial data, and subsequently, for computing volatility. Neural networks have shown promising results in various fields, and interest in finance is also growing. Our approach demonstrates significantly faster computational performance compared to traditional maximum likelihood estimation methods while yielding comparable accuracy in both simulation and empirical studies. Furthermore, we demonstrate the application of this method for real-time volatility measurement, enabling the continuous estimation of financial volatility as new price data keeps coming from the market. |
| Date: | 2023–04 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2304.11883&r=big |
| By: | Csaba Burger (Magyar Nemzeti Bank (the Central Bank of Hungary)); Mihály Berndt (Clarity Consulting Kft) |
| Abstract: | Supervised machine learning methods, in which no error labels are present, are increasingly popular methods for identifying potential data errors. Such algorithms rely on the tenet of a ‘ground truth’ in the data, which in other words assumes correctness in the majority of the cases. Points deviating from such relationships, outliers, are flagged as potential data errors. This paper implements an outlier-based error-spotting algorithm using gradient boosting, and presents a blueprint for the modelling pipeline. More specifically, it underpins three main modelling hypotheses with empirical evidence, which are related to (1) missing value imputation, (2) the loss-function choice and (3) the location of the error. By doing so, it uses a cross sectional view on the loan-to-value and its related columns of the Credit Registry (Hitelregiszter) of the Central Bank of Hungary (MNB), and introduces a set of synthetic error types to test its hypotheses. The paper shows that gradient boosting is not materially impacted by the choice of the imputation method, hence, replacement with a constant, the computationally most efficient, is recommended. Second, the Huber-loss function, which is piecewise quadratic up until the Huber-slope parameter and linear above it, is better suited to cope with outlier values; it is therefore better in capturing data errors. Finally, errors in the target variable are captured best, while errors in the predictors are hardly found at all. These empirical results may generalize to other cases, depending on data specificities, and the modelling pipeline described underscores significant modelling decisions. |
| Keywords: | data quality, machine learning, gradient boosting, central banking, loss functions, missing values |
| JEL: | C5 C81 E58 |
| Date: | 2023 |
| URL: | http://d.repec.org/n?u=RePEc:mnb:opaper:2023/148&r=big |
| By: | Kakuho Furukawa (Bank of Japan); Yoshihiko Hogen (Bank of Japan); Yosuke Kido (Bank of Japan) |
| Abstract: | We analyze labor market tightness and wages for regular workers in Japan, using online job advertisement big data from 2015 to 2022 (approximately 5.8 million samples). The analysis reveals several aspects of the labor market which cannot be captured by official statistics. First, the ratio of job postings matched to job applicants (job-filling rate) has been declining, which suggests that firms may be facing greater difficulties in hiring workers than indicated by macro indicators such as jobs-to-applicants ratio. Second, the decline in the job-filling rate is in part driven by an increase in skill requirements of firms. This is related to the observed acceleration in the accumulation of intangible assets, which has a complementary effect in raising demand for high skilled workers. Third, posted wages are clearly rising under tightening labor market conditions, driven by an increase in demand for high skilled workers. Fourth, an increase in posted wages spills over to average wages of regular workers with some time lag. As for this spillover mechanisms, our empirical results support the existence of (1) a channel in which firms raise wages in order to retain workers as it becomes easier for them to move to higher paying jobs, and (2) a channel in which firms raise wages for fairness consideration as newly hired workers are paid high wages within a firm. |
| Keywords: | Job advertisement; Alternative data; Posted wages; Labor demand; Skill requirement |
| JEL: | J23 J24 J30 |
| URL: | http://d.repec.org/n?u=RePEc:boj:bojwps:wp23e07&r=big |
| By: | Ryuichiro Hashimoto (Bank of Japan); Kakeru Miura (Bank of Japan); Yasunori Yoshizaki (Bank of Japan) |
| Abstract: | Machine learning (ML) has been used increasingly in a wide range of operations at financial institutions. In the field of credit risk management, many financial institutions are starting to apply ML to credit scoring models and default models. In this paper we apply ML to a credit rating classification model. First, we estimate classification models based on both ML and ordinal logistic regression using the same dataset to see how model structure affects the prediction accuracy of models. In addition, we measure variable importance and decompose model predictions using so-called eXplainable AI (XAI) techniques that have been widely used in recent years. The results of our analysis are twofold. First, ML captures more accurately than ordinal logit regression the nonlinear relationships between financial indicators and credit ratings, leading to a significant improvement in prediction accuracy. Second, SHAP (Shapley Additive exPlanations) and PDP (Partial Dependence Plot) show that several financial indicators such as total revenue, total assets turnover, and ICR have a significant impact on firms’ credit quality. Nonlinear relationships between financial indicators and credit rating are also observed: a decrease in ICR below about 2 lowers firms’ credit quality sharply. Our analysis suggests that using XAI while understanding its underlying assumptions improves the low explainability of ML. |
| Keywords: | Credit risk management; Machine learning; Explainability; eXplainable AI (XAI) |
| JEL: | C49 C55 G32 |
| Date: | 2023–04–21 |
| URL: | http://d.repec.org/n?u=RePEc:boj:bojwps:wp23e06&r=big |
| By: | Raj G. Patel; Tomas Dominguez; Mohammad Dib; Samuel Palmer; Andrea Cadarso; Fernando De Lope Contreras; Abdelkader Ratnani; Francisco Gomez Casanova; Senaida Hern\'andez-Santana; \'Alvaro D\'iaz-Fern\'andez; Eva Andr\'es; Jorge Luis-Hita; Escol\'astico S\'anchez-Mart\'inez; Samuel Mugel; Roman Orus |
| Abstract: | The Cheyette model is a quasi-Gaussian volatility interest rate model widely used to price interest rate derivatives such as European and Bermudan Swaptions for which Monte Carlo simulation has become the industry standard. In low dimensions, these approaches provide accurate and robust prices for European Swaptions but, even in this computationally simple setting, they are known to underestimate the value of Bermudan Swaptions when using the state variables as regressors. This is mainly due to the use of a finite number of predetermined basis functions in the regression. Moreover, in high-dimensional settings, these approaches succumb to the Curse of Dimensionality. To address these issues, Deep-learning techniques have been used to solve the backward Stochastic Differential Equation associated with the value process for European and Bermudan Swaptions; however, these methods are constrained by training time and memory. To overcome these limitations, we propose leveraging Tensor Neural Networks as they can provide significant parameter savings while attaining the same accuracy as classical Dense Neural Networks. In this paper we rigorously benchmark the performance of Tensor Neural Networks and Dense Neural Networks for pricing European and Bermudan Swaptions, and we show that Tensor Neural Networks can be trained faster than Dense Neural Networks and provide more accurate and robust prices than their Dense counterparts. |
| Date: | 2023–04 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2304.09750&r=big |
| By: | Simon Briole (CEE-M - Centre d'Economie de l'Environnement - Montpellier - CNRS - Centre National de la Recherche Scientifique - INRAE - Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement - Institut Agro Montpellier - Institut Agro - Institut national d'enseignement supérieur pour l'agriculture, l'alimentation et l'environnement - UM - Université de Montpellier); Augustin Colette (INERIS - Institut National de l'Environnement Industriel et des Risques); Emmanuelle Lavaine (CEE-M - Centre d'Economie de l'Environnement - Montpellier - CNRS - Centre National de la Recherche Scientifique - INRAE - Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement - Institut Agro Montpellier - Institut Agro - Institut national d'enseignement supérieur pour l'agriculture, l'alimentation et l'environnement - UM - Université de Montpellier) |
| Abstract: | While a sharp decline in air pollution has been documented during early Covid-19 lockdown periods, the stability and homogeneity of this effect are still under debate. Building on pollution data with a very high level of resolution, this paper estimates the impact of lockdown policies on P M 2.5 exposure in France over the whole year 2020. Our analyses highlight a surprising and undocumented increase in exposure to particulate pollution during lockdown periods. This result is observed during both lockdown periods, in early spring and late fall, and is robust to several identification strategies and model specifications. Combining administrative datasets with machine learning techniques, this paper also highlights strong spatial heterogeneity in lockdown effects, especially according to long-term pollution exposure. |
| Keywords: | air pollution, P M 2.5, lockdown, spatial heterogeneity, machine learning, Covid-19 |
| Date: | 2023–04–28 |
| URL: | http://d.repec.org/n?u=RePEc:hal:wpaper:hal-04084912&r=big |
| By: | Nozomu Kobayashi; Yoshiyuki Suimon; Koichi Miyamoto; Kosuke Mitarai |
| Abstract: | In this paper we investigate the application of quantum and quantum-inspired machine learning algorithms to stock return predictions. Specifically, we evaluate performance of quantum neural network, an algorithm suited for noisy intermediate-scale quantum computers, and tensor network, a quantum-inspired machine learning algorithm, against classical models such as linear regression and neural networks. To evaluate their abilities, we construct portfolios based on their predictions and measure investment performances. The empirical study on the Japanese stock market shows the tensor network model achieves superior performance compared to classical benchmark models, including linear and neural network models. Though the quantum neural network model attains the lowered risk-adjusted excess return than the classical neural network models over the whole period, both the quantum neural network and tensor network models have superior performances in the latest market environment, which suggests capability of model's capturing non-linearity between input features. |
| Date: | 2023–04 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2304.12501&r=big |
| By: | Athey, Susan (Stanford U); Karlan, Dean (Northwestern U); Palikot, Emil (Stanford U); Yuan, Yuan (Carnegie Mellon U) |
| Abstract: | Online platforms often face challenges being both fair (i.e., non-discriminatory) and efficient (i.e., maximizing revenue). Using computer vision algorithms and observational data from a microlending marketplace, we find that choices made by borrowers creating online profiles impact both of these objectives. We further support this conclusion with a web-based randomized survey experiment. In the experiment, we create profile images using Generative Adversarial Networks that differ in a specific feature and estimate its impact on lender demand. We then counterfactually evaluate alternative platform policies and identify particular approaches to influencing the changeable profile photo features that can ameliorate the fairness-efficiency tension. |
| JEL: | D0 D41 J0 O1 |
| Date: | 2022–11 |
| URL: | http://d.repec.org/n?u=RePEc:ecl:stabus:4071&r=big |
| By: | Ginevra Buratti (Bank of Italy); Alessio D'Ignazio (Bank of Italy) |
| Abstract: | We investigate whether targeting algorithms can improve the effectiveness of financial education programs by identifying the most appropriate recipients in advance. To this end, we use micro-data from approximately 3, 800 individuals who recently participated in a financial education campaign conducted in Italy. Firstly, we employ machine learning (ML) tools to devise a targeting rule that identifies the individuals who should be targeted primarily by a financial education campaign based on easily observable characteristics. Secondly, we simulate a policy scenario and show that pairing a financial education campaign with an ML-based targeting rule enhances its effectiveness. Finally, we discuss a number of conditions that must be met for ML-based targeting to be effectively implemented by policymakers. |
| Keywords: | financial education, machine learning, policy targeting, randomized controlled trials |
| JEL: | C38 I21 G5 |
| Date: | 2023–04 |
| URL: | http://d.repec.org/n?u=RePEc:bdi:opques:qef_765_23&r=big |
| By: | Breen, Casey; Seltzer, Nathan (University of California, Berkeley) |
| Abstract: | How accurately can age of death be predicted using basic sociodemographic characteristics? We test this question using a large-scale administrative dataset combining the complete count 1940 Census with Social Security death records. We fit eight machine learning algorithms using 35 sociodemographic predictors to generate individual-level predictions of age of death for birth cohorts born at the beginning of the 20th century. We find that none of these algorithms are able to explain more than 1.5% of the variation in age of death. Our results suggest mortality is inherently unpredictable and underscore the challenges of using algorithms to predict major life outcomes. |
| Date: | 2023–04–08 |
| URL: | http://d.repec.org/n?u=RePEc:osf:socarx:znsqg&r=big |
| By: | Jean-Charles Bricongne (Centre de recherche de la Banque de France - Banque de France, LEO - Laboratoire d'Économie d'Orleans [2022-...] - UO - Université d'Orléans - UT - Université de Tours - UCA - Université Clermont Auvergne, LIEPP - Laboratoire interdisciplinaire d'évaluation des politiques publiques (Sciences Po) - Sciences Po - Sciences Po); Baptiste Meunier (Centre de recherche de la Banque Centrale européenne - Banque Centrale Européenne, AMSE - Aix-Marseille Sciences Economiques - EHESS - École des hautes études en sciences sociales - AMU - Aix Marseille Université - ECM - École Centrale de Marseille - CNRS - Centre National de la Recherche Scientifique); Sylvain Pouget (Grenoble INP ENSIMAG - École nationale supérieure d'informatique et de mathématiques appliquées - UGA - Université Grenoble Alpes - Grenoble INP - Institut polytechnique de Grenoble - Grenoble Institute of Technology - UGA - Université Grenoble Alpes) |
| Abstract: | While official statistics provide lagged and aggregate information on the housing market, extensive information is available publicly on real-estate websites. By web-scraping them for the UK on a daily basis, this paper extracts a large database from which we build timely and highly granular indicators. One originality of the dataset is to focus on the supply side of the housing market, allowing to compute innovative indicators reflecting the sellers' perspective such as the number of new listings posted or how prices fluctuate over time for existing listings. Matching listing prices in our dataset with transacted prices from the notarial database, using machine learning, also measures the negotiation margin of buyers. During the Covid-19 crisis, these indicators demonstrate the freezing of the market and the "wait-and-see" behaviour of sellers. They also show that listing prices after the lockdown experienced a continued decline in London but increased in other regions. |
| Keywords: | Housing, Real time, Big data, Web-scraping, High frequency, United Kingdom |
| Date: | 2023–03 |
| URL: | http://d.repec.org/n?u=RePEc:hal:spmain:hal-04064185&r=big |
| By: | A. Hennessy, Christopher; Goodhart, C. A. E. |
| Abstract: | We develop a simple structural model to illustrate how penalized regressions generate Goodhart bias when training data are clean but covariates are manipulated at known cost by future agents. With quadratic (extremely steep) manipulation costs, bias is proportional to Ridge (Lasso) penalization. If costs depend on absolute or percentage manipulation, the following algorithm yields manipulation-proof prediction: Within training data, evaluate candidate coefficients at their respective incentive-compatible manipulation configuration. We derive analytical coefficient adjustments: slopes (intercept) shift downward if costs depend on percentage (absolute) manipulation. Statisticians ignoring manipulation costs select socially suboptimal penalization. Model averaging reduces these manipulation costs. |
| JEL: | J1 |
| Date: | 2023–03–21 |
| URL: | http://d.repec.org/n?u=RePEc:ehl:lserod:118656&r=big |
| By: | Li Tang; Chuanli Tang; Qi Fu |
| Abstract: | Accurate and reliable prediction of individual travel mode choices is crucial for developing multi-mode urban transportation systems, conducting transportation planning and formulating traffic demand management strategies. Traditional discrete choice models have dominated the modelling methods for decades yet suffer from strict model assumptions and low prediction accuracy. In recent years, machine learning (ML) models, such as neural networks and boosting models, are widely used by researchers for travel mode choice prediction and have yielded promising results. However, despite the superior prediction performance, a large body of ML methods, especially the branch of neural network models, is also limited by overfitting and tedious model structure determination process. To bridge this gap, this study proposes an enhanced multilayer perceptron (MLP; a neural network) with two hidden layers for travel mode choice prediction; this MLP is enhanced by XGBoost (a boosting method) for feature selection and a grid search method for optimal hidden neurone determination of each hidden layer. The proposed method was trained and tested on a real resident travel diary dataset collected in Chengdu, China. |
| Date: | 2023–04 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2304.12698&r=big |
| By: | Sylvain Barthélémy (TAC Economics, Saint-Hilaire-des-Landes, France); Fabien Rondeau (Univ Rennes, CNRS, CREM – UMR6211, F-35000 Rennes France); Virginie Gautier (TAC Economics and University of Rennes, France.) |
| Abstract: | Currency crises, recurrent events in economic history for developing, emerging and developed countries, generate disastrous economic consequences. This paper proposes an early warning system for currency crises using sophisticated recurrent neural networks like Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). These models were initially used in language processing where they performed well. Such models are increasingly used in forecasting nancial asset prices, including exchange rates, but they have not yet been applied to the prediction of currency crises. As for all recurrent neural networks, they allow to take into account non-linear interactions between variables and the inuence of past data in a dynamic form. For a set of 68 countries including developed, emerging and developing economies over the period 1995-2020, LSTM and GRU outperformed our benchmark models. LSTM and GRU correctly sent continous signals within a two-year warning window to alert 91% of the crises. For LSTM, false signals represent only 14% of the emitted signals compared to 23% for the logistic regression, making them ecient early warning systems for policymakers. |
| Keywords: | currency crises, early warning system, neural network, long short-term memory, gated recurrent unit |
| JEL: | F14 F31 F47 |
| Date: | 2023–04 |
| URL: | http://d.repec.org/n?u=RePEc:tut:cremwp:2023-05&r=big |
| By: | Kapoor, Anuj (Indian Institute of Management, Ahmedabad); Narayanan, Sridhar (Stanford U); Manchanda, Puneet (U of Michigan) |
| Abstract: | Obesity and excess weight are major global health challenges. A number of tech- nological solutions, including mobile apps, have been developed to help people lose weight. Many such applications provide access to human coaches who help consumers set goals, motivate them, answer questions and help them in their weight loss jour- neys. Alternatively, similar services could be provided using AI coaches, which would be cheaper and more scalable than human coaches. In this study, we ask if access to human coaches incrementally affects weight loss outcomes for consumers relative to having AI coaches alone. Our empirical context is a mobile app with two types of subscription plans, those with AI coaches only and those with additional access to human coaches. We compare adopters of the two types of plans on their weight loss achievements. We address potential self-selection into these plans using a matching- based approach that leverages rich behavioral data to find matching consumers on the two types of plans. Our empirical analysis of about 65000 consumers reveals access to human coaches leads to higher weight loss than with AI coaches alone. We document heterogeneity in these differences based on age, gender, and starting BMI of the con- sumers. We also explore potential mechanisms for the human coach impact on weight loss. |
| Date: | 2023–01 |
| URL: | http://d.repec.org/n?u=RePEc:ecl:stabus:4070&r=big |
| By: | Ajit Desai |
| Abstract: | This article provides a curated review of selected papers published in prominent economics journals that use machine learning (ML) tools for research and policy analysis. The review focuses on three key questions: (1) when ML is used in economics, (2) what ML models are commonly preferred, and (3) how they are used for economic applications. The review highlights that ML is particularly used in processing nontraditional and unstructured data, capturing strong nonlinearity, and improving prediction accuracy. Deep learning models are suitable for nontraditional data, whereas ensemble learning models are preferred for traditional datasets. While traditional econometric models may suffice for analyzing low-complexity data, the increasing complexity of economic data due to rapid digitalization and the growing literature suggest that ML is becoming an essential addition to the econometrician's toolbox. |
| Date: | 2023–03 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2304.00086&r=big |
| By: | Hannes Wallimann; Silvio Sticher |
| Abstract: | In railway infrastructure, construction and maintenance is typically procured using competitive procedures such as auctions. However, these procedures only fulfill their purpose - using (taxpayers') money efficiently - if bidders do not collude. Employing a unique dataset of the Swiss Federal Railways, we present two methods in order to detect potential collusion: First, we apply machine learning to screen tender databases for suspicious patterns. Second, we establish a novel category-managers' tool, which allows for sequential and decentralized screening. To the best of our knowledge, we pioneer illustrating the adaption and application of machine-learning based price screens to a railway-infrastructure market. |
| Date: | 2023–04 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2304.11888&r=big |
| By: | Prest, Brian C. (Resources for the Future); Wichman, Casey (Resources for the Future); Palmer, Karen (Resources for the Future) |
| Abstract: | We investigate how well machine learning counterfactual prediction tools can estimate causal treatment effects. We use three prediction algorithms—XGBoost, random forests, and LASSO—to estimate treatment effects using observational data. We compare those results to causal effects from a randomized experiment for electricity customers who faced critical-peak pricing and information treatments. Our results show that each algorithm replicates the true treatment effects, even when using data from treated households only. Additionally, when using both treatment households and nonexperimental comparison households, simpler difference-in-differences methods replicate the experimental benchmark, suggesting little benefit from ML approaches over standard program evaluation methods.Click "Download" above to read the full paper. |
| Date: | 2021–09–29 |
| URL: | http://d.repec.org/n?u=RePEc:rff:dpaper:dp-21-30&r=big |
| By: | Mayank Ratan Bhardwaj (Indian Institute of Science); Jaydeep Pawar (Indian Institute of Science); Abhijnya Bhat (PES University); Deepanshu (Indian Institute of Science); Inavamsi Enaganti (Indian Institute of Science); Kartik Sagar (Indian Institute of Science); Y. Narahari (Indian Institute of Science) |
| Abstract: | Accurate prediction of agricultural crop prices is a crucial input for decision-making by various stakeholders in agriculture: farmers, consumers, retailers, wholesalers, and the Government. These decisions have significant implications including, most importantly, the economic well-being of the farmers. In this paper, our objective is to accurately predict crop prices using historical price information, climate conditions, soil type, location, and other key determinants of crop prices. This is a technically challenging problem, which has been attempted before. In this paper, we propose an innovative deep learning based approach to achieve increased accuracy in price prediction. The proposed approach uses graph neural networks (GNNs) in conjunction with a standard convolutional neural network (CNN) model to exploit geospatial dependencies in prices. Our approach works well with noisy legacy data and produces a performance that is at least 20% better than the results available in the literature. We are able to predict prices up to 30 days ahead. We choose two vegetables, potato (stable price behavior) and tomato (volatile price behavior) and work with noisy public data available from Indian agricultural markets. |
| Date: | 2023–04 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2304.09761&r=big |
| By: | Martina Jakob; Sebastian Heinrich |
| Abstract: | In response to persistent gaps in the availability of survey data, a new strand of research leverages alternative data sources through machine learning to track global development. While previous applications have been successful at predicting outcomes such as wealth, poverty or population density, we show that educational outcomes can be accurately estimated using geo-coded Twitter data and machine learning. Based on various input features, including user and tweet characteristics, topics, spelling mistakes, and network indicators, we can account for ~70 percent of the variation in educational attainment in Mexican municipalities and US counties. |
| Keywords: | machine learning, social media data, education, human capital, indicators, natural language processing |
| JEL: | C53 C80 O11 O15 I21 I25 |
| Date: | 2023–05–05 |
| URL: | http://d.repec.org/n?u=RePEc:bss:wpaper:46&r=big |
| By: | Biggs, Trent; Caviglia-Harris, Jill; Rodrigues Ribeiro, Jime; Ottoni Santiago, Thaís; Sills, Erin; AP West, Thales; Mullan, Katrina |
| Abstract: | We estimate the amount of avoided deforestation due to the use of near-real-time satellite imagery (DETER) to support the Action Plan for the Prevention and Control of Deforestation in the Legal Amazon (PPCDAm), the conservation of indigenous and other protected areas, and compliance with the Brazilian Forest Code (FC). We develop a Directed Acyclical Graph (DAG) that outlines some of the econometric challenges that arise from the role of policy in the estimation of satellite data on deforestation and consider that policy could be a mediator and/or a moderator along this causal chain. We control for other policies that were introduced simultaneously with DETER, and allow for changes in the influences of prices, agricultural settlement, and forest conservation policies on deforestation after near-real-time monitoring was introduced. We find both direct impacts of DETER on deforestation, and indirect impacts via changes in the influences of commodity prices on deforestation. Our counterfactual estimates suggest that 652, 216 km2 of forest was saved from 2000 to 2015 in the Legal Amazon region due to the presence of satellites (43, 481 km2 per year). We estimate that avoided emissions amount to approximately 24 Pg CO2 during our study period. At the municipality level, standardized carbon emission reductions ranged from -1447 to 288, 611 Mg CO2 per km2. |
| Date: | 2022–10–19 |
| URL: | http://d.repec.org/n?u=RePEc:rff:dpaper:dp-22-22&r=big |
| By: | Vafa, Keyon (Columbia U); Palikot, Emil (Stanford U); Du, Tianyu (Stanford U); Kanodia, Ayush (Stanford U); Athey, Susan (Stanford U); Blei, David M. (Columbia U) |
| Abstract: | Labor economists regularly analyze employment data by fitting predictive models to small, carefully constructed longitudinal survey datasets. Although modern machine learning methods offer promise for such problems, these survey datasets are too small to take advantage of them. In recent years large datasets of online resumes have also become available, providing data about the career trajectories of millions of individuals. However, standard econometric models cannot take advantage of their scale or incorporate them into the analysis of survey data. To this end we develop CAREER, a transformer-based model that uses transfer learning to learn representations of job sequences. CAREER is first fit to large, passively-collected resume data and then fine-tuned to smaller, better-curated datasets for economic inferences. We fit CAREER to a dataset of 24 million job sequences from resumes, and fine-tune its representations on longitudinal survey datasets. We find that CAREER forms accurate predictions of job sequences on three widely-used economics datasets. We further find that CAREER can be used to form good predictions of other downstream variables; incorporating CAREER into a wage model provides better predictions than the econometric models currently in use. |
| Date: | 2022–10 |
| URL: | http://d.repec.org/n?u=RePEc:ecl:stabus:4074&r=big |
| By: | Jiwook Kim; Minhyeok Lee |
| Abstract: | In financial engineering, portfolio optimization has been of consistent interest. Portfolio optimization is a process of modulating asset distributions to maximize expected returns and minimize risks. To obtain the expected returns, deep learning models have been explored in recent years. However, due to the deterministic nature of the models, it is difficult to consider the risk of portfolios because conventional deep learning models do not know how reliable their predictions can be. To address this limitation, this paper proposes a probabilistic model, namely predictive auxiliary classifier generative adversarial networks (PredACGAN). The proposed PredACGAN utilizes the characteristic of the ACGAN framework in which the output of the generator forms a distribution. While ACGAN has not been employed for predictive models and is generally utilized for image sample generation, this paper proposes a method to use the ACGAN structure for a probabilistic and predictive model. Additionally, an algorithm to use the risk measurement obtained by PredACGAN is proposed. In the algorithm, the assets that are predicted to be at high risk are eliminated from the investment universe at the rebalancing moment. Therefore, PredACGAN considers both return and risk to optimize portfolios. The proposed algorithm and PredACGAN have been evaluated with daily close price data of S&P 500 from 1990 to 2020. Experimental scenarios are assumed to rebalance the portfolios monthly according to predictions and risk measures with PredACGAN. As a result, a portfolio using PredACGAN exhibits 9.123% yearly returns and a Sharpe ratio of 1.054, while a portfolio without considering risk measures shows 1.024% yearly returns and a Sharpe ratio of 0.236 in the same scenario. Also, the maximum drawdown of the proposed portfolio is lower than the portfolio without PredACGAN. |
| Date: | 2023–04 |
| URL: | http://d.repec.org/n?u=RePEc:arx:papers:2304.11856&r=big |