Machine Learning: Finance

Part: Playbooks · Chapter: Playbook: Machine Learning in Finance · Edition: 2026-07-12
Keywords: markets, features, backtest, leakage, risk
Glossary: backtest, leakage, alpha, risk — see GLOSSARY

At a glance

This playbook records machine-learning methods as applied to financial time series.
It is orientation for reading those methods.
It is not financial, legal or investment advice, and it is not a trading system.

Online references (live)

Offline snapshots (link-rot shield)

Offline snapshot omitted in this web edition. Offline snapshot · Quantitative finance · https://en.wikipedia.org/wiki/Quantitative_analysis_(finance) · retrieved 2026-07-12

Quantitative analysis (finance)

Offline snapshot omitted in this web edition. Offline snapshot · Backtesting · https://en.wikipedia.org/wiki/Backtesting · retrieved 2026-07-12

Backtesting

Applications at Keyne Digital Assets

Time-series and predictive models appear in many monitoring settings, not only in markets. We read this chapter for that methodological frame. Situation Monitor does not execute trades, and we do not recommend any asset or strategy.

Introduction

Building trading bots that utilize AI and graphic analysis involves a blend of financial expertise, statistical modeling, and computational technology. Here are some well-known techniques often used to support trading strategies:

Machine Learning Algorithms: These can range from simple linear regression models for trend forecasting to complex neural networks for pattern recognition. Machine learning models can process massive datasets to identify market trends and make predictions. Some popular algorithms include:
* Support Vector Machines (SVM)
* Random Forests
* Gradient Boosting Machines (GBM)
* Deep Learning (e.g., Convolutional Neural Networks for image-based pattern recognition)
Time Series Analysis: This involves analyzing market data points, like stock prices, at successive time intervals to forecast future movements. Techniques include:
* ARIMA (AutoRegressive Integrated Moving Average)
* GARCH (Generalized Autoregressive Conditional Heteroskedasticity) models for volatility prediction
Natural Language Processing (NLP): Bots can analyze news articles, social media, or financial reports to gauge market sentiment. Techniques include:
* Sentiment Analysis
* Topic Modeling
Technical Analysis: Involves analyzing statistical trends gathered from trading activity, such as price movement and volume. Chart patterns and technical indicators are used, such as:
* Moving Averages
* MACD (Moving Average Convergence Divergence)
* RSI (Relative Strength Index)
* Fibonacci Retracements
Graphical Analysis: Also known as chart analysis, where patterns on stock charts are used to forecast market movements. Techniques include:
* Candlestick Patterns
* Head and Shoulders Patterns
* Cup-and-Handle Patterns
Quantitative Analysis: Involves the application of mathematical and statistical techniques to determine trading opportunities. Techniques include:
* Monte Carlo Simulations
* Mean Reversion Strategies
* Statistical Arbitrage
Algorithmic Execution Strategies: These focus on the execution of trades rather than prediction. They break up a large order into smaller pieces and execute them algorithmically to minimize market impact. Examples include:
* VWAP (Volume Weighted Average Price)
* TWAP (Time Weighted Average Price)
* Iceberg Orders
Reinforcement Learning: A type of machine learning where algorithms learn to make trading decisions through trial and error. Algorithms are rewarded for profitable strategies.
Sentiment Indicators: Derived from social media and other public communications, these indicators assess the mood of the market towards a particular stock or the market in general.
High-Frequency Trading (HFT) Algorithms: Use complex algorithms to analyze multiple markets and execute orders based on market conditions. They often involve:
- Latency Arbitrage
- Market Making
- Statistical Arbitrage

Each of these techniques can be powerful, but they also have limitations and require careful risk management. Additionally, the effectiveness of a trading bot depends on the quality of the underlying data, the appropriateness of the algorithms for the market conditions, and the speed of execution. Moreover, regulatory compliance and ethical considerations must be taken into account when deploying trading bots in real markets.

Linear regression ML model

Developing a well-established strategy for a machine learning (ML) linear regression model in trading involves several key steps, from data selection and preprocessing to model development and backtesting. Let’s outline a strategy that includes real-time data from trusted sources other than just price over days.

1. Data Selection and Sources

Features Beyond Price: Incorporate features like trading volume, bid-ask spreads, historical volatility, moving averages, and relative strength index (RSI).
Fundamental Data: Economic indicators, company earnings reports, and interest rates can also be influential.
Alternative Data: Social media sentiment, news sentiment analysis, and Google Trends data can provide insights into market sentiment.
Real-Time Data Sources: Financial data APIs like Alpha Vantage, Yahoo Finance, or Quandl offer real-time and historical market data. Bloomberg and Reuters are also reputable for real-time financial news.

2. Data Preprocessing

Normalization: Scale the data to ensure that different features contribute equally to the model.
Handling Missing Data: Fill or remove missing values in a way that doesn’t introduce bias.
Feature Engineering: Create new features that might capture market dynamics better, like technical indicators.

3. Model Development

Linear Regression Model: Start with a simple linear regression model to establish a baseline.
Regularization Techniques: Implement Lasso or Ridge regression to prevent overfitting and improve generalization.
Model Validation: Use cross-validation techniques to validate the model’s performance.

4. Incorporating Real-Time Data

Data Streaming: Set up a system for streaming real-time data into your model.
Model Updating: Implement a method to update the model as new data comes in, possibly using techniques like online learning or periodic retraining.

5. Risk Management

Stop-Loss and Take-Profit: Set clear rules for stop-loss and take-profit to manage risks.
Position Sizing: Use prudent position sizing to avoid significant losses.

6. Backtesting

Historical Data: Test the strategy on historical data to understand its performance across different market conditions.
Metrics: Evaluate using metrics like Sharpe ratio, maximum drawdown, and return on investment.

7. Execution

Algorithmic Trading: Integrate the model with a trading platform for automated execution.
Monitoring: Continuously monitor the model’s performance and adjust as needed.

8. Compliance and Ethical Considerations

Regulatory Compliance: Ensure the strategy complies with all financial market regulations.
Data Privacy: Respect data privacy and ethical considerations, especially when using alternative data sources.

I’ll provide examples for two different scenarios in each case: the structure of the LSTM model, data preprocessing for the LSTM model, and the method used for making predictions.

LSTM models

Option 1: Simple LSTM Model

from keras.models import Sequential
from keras.layers import LSTM, Dense

model = Sequential()
model.add(LSTM(units=50, return_sequences=False, input_shape=(window_size, feature_count)))
model.add(Dense(units=1))

model.compile(optimizer='adam', loss='mean_squared_error')

This is a simple LSTM model with one LSTM layer and one output layer. units=50 indicates the number of LSTM units, return_sequences=False means the LSTM layer outputs a single value, and input_shape depends on the window size and number of features.

Option 2: Stacked LSTM Model

from keras.models import Sequential
from keras.layers import LSTM, Dense

model = Sequential()
model.add(LSTM(units=50, return_sequences=True, input_shape=(window_size, feature_count)))
model.add(LSTM(units=50, return_sequences=False))
model.add(Dense(units=1))

model.compile(optimizer='adam', loss='mean_squared_error')

This model has two LSTM layers stacked together. The first LSTM layer has return_sequences=True to pass the sequence to the next layer.

Scaling data

Choosing the best scaling method for time series data related to stocks or cryptocurrencies is an important step in preprocessing for various machine learning and data analysis tasks. The choice of scaler depends on the nature of your data and the specific requirements of your analysis or model. Here are some common scaling methods:

Standard Scaler (Z-score normalization): This method transforms the data to have a mean of 0 and a standard deviation of 1. It’s useful when your data follows a Gaussian distribution, but it’s sensitive to outliers.
Min-Max Scaler: This scaler transforms the data to a specified range, typically [0, 1]. It’s a good choice when you need to maintain the shape of the original distribution and don’t have outliers.
Robust Scaler: This method is similar to the Standard Scaler but uses the median and the interquartile range, making it robust to outliers. It’s useful when you have data with outliers.
Log Transformation: Applying a log transformation can help stabilize the variance of a time series, especially in financial data where there can be multiplicative effects.
Box-Cox / Yeo-Johnson Transformation: These are generalized power transformations that work for different types of data distributions. They can stabilize variance and make the data more normal (Gaussian).
Quantile Transformer: This method transforms the features to follow a uniform or a normal distribution, and therefore, it smoothens out unusual distributions.
Normalization by Decimal Scaling: This involves shifting the decimal point of values. It’s a simpler method but less commonly used in sophisticated analyses.
Custom Scalers: Sometimes, a custom scaling method tailored to the specific characteristics of your data can be the best approach.

For stock and cryptocurrency time series, you typically deal with non-stationary and often volatile data. It’s crucial to consider the following:

Nature of Data: If the data contains outliers, robust scalers are preferred. If the data has a normal distribution, standard scaling might be appropriate.
Model Requirements: Some models require data to be normally distributed; others might need data within a certain range.
Purpose of Analysis: For instance, log transformation can be particularly useful for trend analysis in time series.

It’s often recommended to experiment with different scalers and evaluate their performance based on your specific use case. Cross-validation and performance metrics can help determine the best approach for your particular dataset and objectives.

Data Preprocessing for LSTM Model

Option 1: Single Time Step

window_size = 1
X, y = [], []
for i in range(len(data) - window_size):
    X.append(data[i:(i + window_size), 0])
    y.append(data[i + window_size, 0])
X, y = np.array(X), np.array(y)

This code snippet creates input sequences where each input sequence contains only 1 time step.

Option 2: Multiple Time Steps

window_size = 5
X, y = [], []
for i in range(len(data) - window_size):
    X.append(data[i:(i + window_size), 0])
    y.append(data[i + window_size, 0])
X, y = np.array(X), np.array(y)

This snippet prepares the data with a window size of 5, meaning each input sequence contains 5 time steps.

Prediction Method

Option 1: Direct Approach

# Predict the next value
next_value = model.predict(X[-1].reshape(1, window_size, feature_count))

Using actual historical data for the next prediction

new_input = np.append(X[-1][1:], next_value).reshape(1, window_size, feature_count)
next_next_value = model.predict(new_input)

Here, the model makes a prediction based on the actual last known data point. For subsequent predictions, it still uses actual data where available.

Option 2: Iterative Approach

# Predict the next value
next_value = model.predict(X[-1].reshape(1, window_size, feature_count))

Using the prediction for the next prediction

new_input = np.append(X[-1][1:], next_value)
for i in range(number_of_future_steps):
    next_value = model.predict(new_input.reshape(1, window_size, feature_count))
    new_input = np.append(new_input[1:], next_value)

In this approach, each prediction is based on the previous prediction, iterating forward for a specified number of future steps.

These examples can be adjusted based on the specific requirements of your time series forecasting task.

Layers

It’s possible to design a neural network, particularly an LSTM (Long Short-Term Memory) network, that incorporates multiple layers with different sets of features, including raw data like date, close price, and volume, as well as layers with combinations of different technical indicators. This approach is often used to capture various aspects of the data, like raw price movements and derived indicators, which might provide additional insights for the prediction task. Here’s a conceptual overview of how you could structure such a model:

Layer 1: Raw Data

Features: Date, Close Price, and Volume.
Purpose: This layer can directly capture the raw market movements.
Implementation: You might convert the date into a numerical format (like timestamps or separate components like year, month, day) to use it as a feature.

Layer 2: Technical Indicators Set 1

Features: A set of selected technical indicators (e.g., Moving Averages, RSI).
Purpose: To capture trends and momentum in the data.
Implementation: These indicators can be calculated using libraries like ta (Technical Analysis library in Python) and appended as additional features.

Layer 3: Technical Indicators Set 2

Features: Another set of technical indicators (e.g., MACD, Bollinger Bands).
Purpose: To provide signals on market volatility and potential reversals.
Implementation: Similar to the second layer, but with a different set of indicators.

Model Structure

You could structure the LSTM model to process these layers either sequentially or in parallel:

Sequential Approach: Each LSTM layer processes one set of features, and the output of one layer is fed into the next. This way, the higher layers can learn to interpret the outputs of the previous layers.

python
from keras.models import Sequential
from keras.layers import LSTM, Dense

model = Sequential()
model.add(LSTM(units=50, input_shape=(time_steps, features_layer_1)))
model.add(LSTM(units=50, input_shape=(time_steps, features_layer_2)))
model.add(LSTM(units=50, input_shape=(time_steps, features_layer_3)))
model.add(Dense(units=1))

Parallel Approach: Each set of features is processed by a separate LSTM branch, and their outputs are concatenated before making a prediction. This approach allows each branch to independently learn from its respective set of features.

python
from keras.models import Model
from keras.layers import Input, LSTM, Dense, concatenate

Define three inputs

input1 = Input(shape=(time_steps, features_layer_1))
input2 = Input(shape=(time_steps, features_layer_2))
input3 = Input(shape=(time_steps, features_layer_3))

LSTM layers for each input

lstm1 = LSTM(units=50)(input1)
lstm2 = LSTM(units=50)(input2)
lstm3 = LSTM(units=50)(input3)

Concatenate the outputs of the LSTM layers

concatenated = concatenate([lstm1, lstm2, lstm3])

Dense layer for prediction

output = Dense(units=1)(concatenated)

model = Model(inputs=[input1, input2, input3], outputs=output)

In both approaches, it’s crucial to ensure that the data corresponding to each layer is correctly aligned in terms of time steps. This setup can potentially enhance the model’s ability to learn from both raw data and derived features, thus potentially improving its predictive accuracy. However, it also increases the model’s complexity, so careful training, validation, and testing are essential to avoid overfitting and ensure robust performance.

Conclusion

This strategy provides a structured approach to developing an ML linear regression model for trading, incorporating a variety of data sources and techniques. However, it’s important to note that trading involves significant risk, and ML models are no exception. Continuous evaluation and adaptation are key to maintaining a successful trading strategy.

Part index

The scientific map, the equations, and the bibliographic spine for this part of the Knowledge Base are collected in the part indexes:

General · Engineering · Computer science · Playbooks
Indexes · Glossary · SUMMARY

Wikipedia snapshots in the chapter card offer offline orientation; the part indexes carry the full citation trails.