Global AI and Data Science

View Only

Time-series data analysis using Deep Learning

By Randhir Singh posted Wed May 22, 2024 07:34 PM

Time-series data analysis using Deep Learning¶

In this blog, we'll analyze stock prices of a company with the aim of predicting stock prices based on historical prices.

The stock price levels have a structure that we want to exploit. Time-series is usually modeled in a recursive way, typically using auto-regressive models, which are linear models that don't consider non-linearity in the data.

In particular, we illustrate the use of two deep learning techniques for analyzing time-series data - Multi-Layer Perceptron (MLP) and Convolutional Neural Network (CNN) to analyze historical stock prices of a company and use the trained model to predict stock prices. The main goal is to learn how to use these tools for time-series prediction.

Let's start by installing or importing required Python packages. We'll use Yahoo Finance to download stock prices and Tensorflow to implement MLP and CNN.

!pip install pyts

Requirement already satisfied: pyts in /usr/local/lib/python3.10/dist-packages (0.13.0)
Requirement already satisfied: numpy>=1.22.4 in /usr/local/lib/python3.10/dist-packages (from pyts) (1.25.2)
Requirement already satisfied: scipy>=1.8.1 in /usr/local/lib/python3.10/dist-packages (from pyts) (1.11.4)
Requirement already satisfied: scikit-learn>=1.2.0 in /usr/local/lib/python3.10/dist-packages (from pyts) (1.2.2)
Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from pyts) (1.4.2)
Requirement already satisfied: numba>=0.55.2 in /usr/local/lib/python3.10/dist-packages (from pyts) (0.58.1)
Requirement already satisfied: llvmlite<0.42,>=0.41.0dev0 in /usr/local/lib/python3.10/dist-packages (from numba>=0.55.2->pyts) (0.41.1)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn>=1.2.0->pyts) (3.5.0)

import numpy as np
import pandas as pd
import yfinance as yf
import seaborn as sns
import gc
import os
import random
import copy
import matplotlib.pyplot as plt
import statsmodels
from statsmodels.tsa.stattools import adfuller
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Dropout
from keras.callbacks import EarlyStopping

from pyts.image import GramianAngularField

We'll analyze the closing price of Apple (AAPL) stocks. Let's collect data from past 2000 trading days and visualize the raw data.

# Load stock price data
df = yf.download("AAPL", start="2016-06-01", end="2024-05-21")
df=df.drop(columns=['Open','High','Low','Volume', 'Adj Close'])

# Plot the original time series
df.plot(figsize=(15,6))
plt.title('Stock Price Time Series')
plt.xlabel('Date')
plt.ylabel('Price (USD)')
plt.grid(True)
plt.show()

[*********************100%%**********************]  1 of 1 completed

Let's check the summary statistics.

# Summary statistics
print("Summary Statistics:")
print(df.describe())

Summary Statistics:
             Close
count  2006.000000
mean     98.450604
std      56.856812
min      23.010000
25%      43.598126
50%      80.936249
75%     150.799999
max     198.110001

Usually univariate time series analysis is done on a stationary series, so first off we'll check for stationarity. To check stationarity visually, we plot ACF and PACF for different lags. The blue shaded region shows the confidence intervals.

# ACF and PACF plots for persistence

plt.figure(figsize=(12, 6))
plt.subplot(2, 1, 1)
plot_acf(df, ax=plt.gca(), lags=50)
plt.title('Autocorrelation Function (ACF)')
plt.subplot(2, 1, 2)
plot_pacf(df, ax=plt.gca(), lags=50)
plt.title('Partial Autocorrelation Function (PACF)')
plt.tight_layout()
plt.show()

It is evident from the plot that the time-series is non-stationary. We can conclude this because the lags of the ACF are significant.

After visual inspection, let us try to confirm our conclusion using hypothesis testing method - ADF test, to test for stationarity.

# Augmented Dickey-Fuller (ADF) test for stationarity
adf_result = adfuller(df)
print("\nADF Test Results:")
print("ADF Statistic:", adf_result[0])
print("p-value:", adf_result[1])
print("Critical Values:")
for key, value in adf_result[4].items():
    print(f"\t{key}: {value}")

ADF Test Results:
ADF Statistic: -0.4605141594598856
p-value: 0.8994943104821944
Critical Values:
	1%: -3.433630425730241
	5%: -2.862989113874071
	10%: -2.5675414465504156

The ADF statistic is larger than the critical values at different level of significance. We cannot reject the null hypothesis of non-stationarity.

Let's use the logarithm of the original series to see if transformed series is stationary.

df_log = pd.DataFrame(np.log(df['Close']),index=df.index)
plt.figure(figsize=(15,6))
plt.plot(df_log)
plt.title('Log Close')
plt.show()

# Augmented Dickey-Fuller (ADF) test for stationarity
adf_result = adfuller(df_log)
print("\nADF Test Results:")
print("ADF Statistic:", adf_result[0])
print("p-value:", adf_result[1])
print("Critical Values:")
for key, value in adf_result[4].items():
    print(f"\t{key}: {value}")

ADF Test Results:
ADF Statistic: -1.2776245051912318
p-value: 0.6393459548923692
Critical Values:
	1%: -3.433630425730241
	5%: -2.862989113874071
	10%: -2.5675414465504156

The log of the series is also non-stationary. Some common techniques to make time series data stationary:

Differencing: Take the difference between consecutive observations.
Transformation: Apply transformations such as logarithmic transformation
Fractional Differencing: Partially doing differencing to achieve stationarity

Differencing¶

Let's now try to difference the series or the log of the series.

df['Returns']=df['Close']-df['Close'].shift(1)
df_log['Log_Returns']=df_log['Close']-df_log['Close'].shift(1)
plt.figure(figsize=(15,6))
plt.subplot(1,2,1)
plt.plot(df['Returns'])
plt.title('Returns')
plt.subplot(1,2,2)
plt.plot(df_log['Log_Returns'],'g-')
plt.title('Log Returns')
plt.show()

Let's check both of our new series through the ADF test.

adf_result_returns = adfuller(df['Returns'].dropna())
print("\nADF Test Results for Returns:")
print("ADF Statistic:", adf_result_returns[0])
print("p-value:", adf_result_returns[1])
print("Critical Values:")
for key, value in adf_result_returns[4].items():
    print(f"\t{key}: {value}")


adf_result_log_returns = adfuller(df_log['Log_Returns'].dropna())
print("\nADF Test Results for Log_Returns:")
print("ADF Statistic:", adf_result_log_returns[0])
print("p-value:", adf_result_log_returns[1])
print("Critical Values:")
for key, value in adf_result_log_returns[4].items():
    print(f"\t{key}: {value}")

ADF Test Results for Returns:
ADF Statistic: -14.606628895391774
p-value: 4.084929825598094e-27
Critical Values:
	1%: -3.433630425730241
	5%: -2.862989113874071
	10%: -2.5675414465504156

ADF Test Results for Log_Returns:
ADF Statistic: -14.068038529279898
p-value: 2.974401974504954e-26
Critical Values:
	1%: -3.433630425730241
	5%: -2.862989113874071
	10%: -2.5675414465504156

We have stationary series that we can now work with for our forecasts.

Fractional Differencing¶

Next, let's use an approach called fractional differencing. Using fractionally differenced variables as dependent variables or features for time series modelling could improve predictive performance in many cases because the time series is allowed to retain more memory while still being stationary.

def getWeights(d,lags):
    # return the weights from the series expansion of the differencing operator
    # for real orders d and up to lags coefficients
    w=[1]
    for k in range(1,lags):
        w.append(-w[-1]*((d-k+1))/k)
    w=np.array(w).reshape(-1,1)
    return w
def plotWeights(dRange, lags, numberPlots):
    weights=pd.DataFrame(np.zeros((lags, numberPlots)))
    interval=np.linspace(dRange[0],dRange[1],numberPlots)
    for i, diff_order in enumerate(interval):
        weights[i]=getWeights(diff_order,lags)
    weights.columns = [round(x,2) for x in interval]
    fig=weights.plot(figsize=(15,6))
    plt.legend(title='Order of differencing')
    plt.title('Lag coefficients for various orders of differencing')
    plt.xlabel('lag coefficients')
    #plt.grid(False)
    plt.show()
def ts_differencing(series, order, lag_cutoff):
    # return the time series resulting from (fractional) differencing
    # for real orders order up to lag_cutoff coefficients

    weights=getWeights(order, lag_cutoff)
    res=0
    for k in range(lag_cutoff):
        res += weights[k]*series.shift(k).fillna(0)
    return res[lag_cutoff:]

df['Fractional'] = ts_differencing(df['Close'],0.9,20)
df_log['Fractional'] = ts_differencing(df_log['Close'],0.9,20)

differences=[0.9]
fig, axs = plt.subplots(len(differences),2,figsize=(15,6))
for i in range(0,len(differences)):
    axs[0].plot(df['Fractional'])
    axs[0].set_title('Original series with d='+str(differences[i]))
    axs[1].plot(df_log['Fractional'],'g-')
    axs[1].set_title('Logarithmic series with d='+str(differences[i]))
    plt.subplots_adjust(bottom=0.01) #increasing space between plots for aestethics

Let's check both of our new series through the ADF test.

adf_result_fractional = adfuller(df['Fractional'].dropna())
print("\nADF Test Results for Fractional:")
print("ADF Statistic:", adf_result_fractional[0])
print("p-value:", adf_result_fractional[1])
print("Critical Values:")
for key, value in adf_result_fractional[4].items():
    print(f"\t{key}: {value}")


adf_result_log_fractional = adfuller(df_log['Fractional'].dropna())
print("\nADF Test Results for Fractional:")
print("ADF Statistic:", adf_result_log_fractional[0])
print("p-value:", adf_result_log_fractional[1])
print("Critical Values:")
for key, value in adf_result_log_fractional[4].items():
    print(f"\t{key}: {value}")

ADF Test Results for Fractional:
ADF Statistic: -7.401482908989515
p-value: 7.534908468166931e-11
Critical Values:
	1%: -3.4336771595431106
	5%: -2.863009746829746
	10%: -2.5675524325901415

ADF Test Results for Fractional:
ADF Statistic: -6.328209788530858
p-value: 2.9503178659131254e-08
Critical Values:
	1%: -3.4336788532475775
	5%: -2.8630104945933974
	10%: -2.567552830738709

After fractional differencing, the series is stationary.

The data frame contains three series - the closing price, the daily returns and the fractional differences. Each of the three series will be used for forecasting individually.

df.tail()

	Close	Returns	Fractional
Date
2024-05-14	187.429993	1.149994	2.954432
2024-05-15	189.720001	2.290009	4.046171
2024-05-16	189.839996	0.119995	1.981708
2024-05-17	189.869995	0.029999	1.736780
2024-05-20	191.039993	1.169998	2.794917

Multi-Layer Perceptron model (MLP or Artificial Neural Network)¶

def train_test_data(target_variable):
  """
  Return the training and testing data for the target variable. The target variable
  is one of the series - Closing price, daily returns or fractional differences.
  The data is split into 80% training and 20% testing based on time. The first 80%
  of the data is used for training and the last 20% for testing.
  """
  df_copy = df.loc[:,df.columns.isin([target_variable])]

  # Creating input features by shifting the target variable
  look_back = 5  # Number of past time steps to consider
  for i in range(1, look_back+1):
    df_copy[f'lag_{i}'] = df_copy[target_variable].shift(i)

  df_copy.dropna(inplace=True)

  scaler = MinMaxScaler()
  df_copy[df_copy.columns] = scaler.fit_transform(df_copy[df_copy.columns])

  X = df_copy.iloc[:, 1:].values  # Input features
  y = df_copy[target_variable].values  # Target variable

  # Calculate the index to split the data
  split_index = int(len(df_copy) * 0.8)  # Use 80% of the data for training

  # Split the data into training and testing sets
  train_data = df_copy.iloc[:split_index]
  test_data = df_copy.iloc[split_index:]
  return train_data, test_data

def mlp(target_variable, train_data, test_data):
  """
  Method to train, test and plot the results of a Multi-Layer Perceptron (MLP) model
  for predicting the target variable.
  """
  # Extract input features and target variable for training and testing sets
  X_train = train_data.iloc[:, 1:].values
  y_train = train_data[target_variable].values
  X_test = test_data.iloc[:, 1:].values
  y_test = test_data[target_variable].values

  hp_lr = 1e-5  # Learning rate
  adam = tf.keras.optimizers.Adam(learning_rate=hp_lr)  # Adam optimizer
  model = get_mlp_model(X_train)
  es = EarlyStopping(
    monitor="val_loss", mode="min", verbose=1, patience=10, restore_best_weights=True
  )
  model.compile(optimizer=adam, loss='mean_squared_error', metrics=['mae'])

  model.fit(X_train, y_train, epochs=500, batch_size=32, validation_split=0.2, callbacks=[es])

  loss, mae = model.evaluate(X_test, y_test)
  print("Test Loss:", loss)
  print("Test MAE:", mae)

  predictions = model.predict(X_test)

  plt.figure(figsize=(20, 6))
  plt.plot(y_test, label='Actual')
  plt.plot(predictions, label='Predicted')
  plt.title(f'Actual vs. Predicted Values ({target_variable})')
  plt.xlabel('Time')
  plt.ylabel('Close Price')
  plt.legend()
  plt.grid(True)
  plt.show()

Let's define a MLP model that we'll train on historical stock prices.

def get_mlp_model(X_train):
  """
  Method to define a MLP model for predicting the target variable.
  The model consists of a input layer, a dense hidden layer with 64 neurons, a dropout layer,
  and a dense output layer.
  """
  mlp_model = Sequential([
    Dense(64, activation='relu', input_shape=(X_train.shape[1],)),
    Dropout(0.2),
    Dense(64, activation='relu'),
    Dropout(0.2),
    Dense(1)  # Output layer with 1 neuron for regression
  ])
  mlp_model.summary()
  return mlp_model

Predicting Price Levels¶

First, let's try out our model for forecasting the original price levels, the non-stationary data.

target_variable = 'Close'
train_data, test_data = train_test_data(target_variable)
mlp(target_variable, train_data, test_data)

Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_3 (Dense)             (None, 64)                384       
                                                                 
 dropout_2 (Dropout)         (None, 64)                0         
                                                                 
 dense_4 (Dense)             (None, 64)                4160      
                                                                 
 dropout_3 (Dropout)         (None, 64)                0         
                                                                 
 dense_5 (Dense)             (None, 1)                 65        
                                                                 
=================================================================
Total params: 4609 (18.00 KB)
Trainable params: 4609 (18.00 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

<ipython-input-35-db7a072d7ce2>:13: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  df_copy[f'lag_{i}'] = df_copy[target_variable].shift(i)

Epoch 1/500
40/40 [==============================] - 1s 10ms/step - loss: 0.1703 - mae: 0.3101 - val_loss: 1.0274 - val_mae: 1.0096
Epoch 2/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1693 - mae: 0.3079 - val_loss: 0.9906 - val_mae: 0.9913
Epoch 3/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1565 - mae: 0.2951 - val_loss: 0.9561 - val_mae: 0.9738
Epoch 4/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1528 - mae: 0.2894 - val_loss: 0.9226 - val_mae: 0.9566
Epoch 5/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1484 - mae: 0.2845 - val_loss: 0.8883 - val_mae: 0.9386
Epoch 6/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1386 - mae: 0.2750 - val_loss: 0.8554 - val_mae: 0.9211
Epoch 7/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1366 - mae: 0.2707 - val_loss: 0.8234 - val_mae: 0.9037
Epoch 8/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1309 - mae: 0.2636 - val_loss: 0.7925 - val_mae: 0.8865
Epoch 9/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1249 - mae: 0.2573 - val_loss: 0.7624 - val_mae: 0.8694
Epoch 10/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1188 - mae: 0.2499 - val_loss: 0.7339 - val_mae: 0.8530
Epoch 11/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1144 - mae: 0.2449 - val_loss: 0.7063 - val_mae: 0.8368
Epoch 12/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1146 - mae: 0.2422 - val_loss: 0.6799 - val_mae: 0.8210
Epoch 13/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1026 - mae: 0.2288 - val_loss: 0.6542 - val_mae: 0.8053
Epoch 14/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1013 - mae: 0.2272 - val_loss: 0.6299 - val_mae: 0.7902
Epoch 15/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0975 - mae: 0.2215 - val_loss: 0.6059 - val_mae: 0.7750
Epoch 16/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0943 - mae: 0.2163 - val_loss: 0.5828 - val_mae: 0.7600
Epoch 17/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0903 - mae: 0.2109 - val_loss: 0.5608 - val_mae: 0.7455
Epoch 18/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0868 - mae: 0.2063 - val_loss: 0.5398 - val_mae: 0.7314
Epoch 19/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0847 - mae: 0.2021 - val_loss: 0.5194 - val_mae: 0.7174
Epoch 20/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0767 - mae: 0.1925 - val_loss: 0.5004 - val_mae: 0.7041
Epoch 21/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0763 - mae: 0.1902 - val_loss: 0.4812 - val_mae: 0.6905
Epoch 22/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0735 - mae: 0.1863 - val_loss: 0.4628 - val_mae: 0.6771
Epoch 23/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0693 - mae: 0.1796 - val_loss: 0.4454 - val_mae: 0.6642
Epoch 24/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0680 - mae: 0.1771 - val_loss: 0.4286 - val_mae: 0.6515
Epoch 25/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0634 - mae: 0.1706 - val_loss: 0.4128 - val_mae: 0.6394
Epoch 26/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0621 - mae: 0.1680 - val_loss: 0.3970 - val_mae: 0.6270
Epoch 27/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0568 - mae: 0.1600 - val_loss: 0.3820 - val_mae: 0.6150
Epoch 28/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0538 - mae: 0.1545 - val_loss: 0.3676 - val_mae: 0.6033
Epoch 29/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0531 - mae: 0.1522 - val_loss: 0.3531 - val_mae: 0.5913
Epoch 30/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0508 - mae: 0.1472 - val_loss: 0.3393 - val_mae: 0.5795
Epoch 31/500
40/40 [==============================] - 0s 7ms/step - loss: 0.0485 - mae: 0.1442 - val_loss: 0.3255 - val_mae: 0.5676
Epoch 32/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0469 - mae: 0.1395 - val_loss: 0.3123 - val_mae: 0.5559
Epoch 33/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0466 - mae: 0.1373 - val_loss: 0.2987 - val_mae: 0.5436
Epoch 34/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0414 - mae: 0.1311 - val_loss: 0.2848 - val_mae: 0.5307
Epoch 35/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0416 - mae: 0.1293 - val_loss: 0.2718 - val_mae: 0.5184
Epoch 36/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0366 - mae: 0.1210 - val_loss: 0.2591 - val_mae: 0.5061
Epoch 37/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0352 - mae: 0.1182 - val_loss: 0.2469 - val_mae: 0.4940
Epoch 38/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0339 - mae: 0.1166 - val_loss: 0.2351 - val_mae: 0.4820
Epoch 39/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0320 - mae: 0.1127 - val_loss: 0.2235 - val_mae: 0.4700
Epoch 40/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0320 - mae: 0.1113 - val_loss: 0.2129 - val_mae: 0.4586
Epoch 41/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0314 - mae: 0.1102 - val_loss: 0.2024 - val_mae: 0.4471
Epoch 42/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0285 - mae: 0.1062 - val_loss: 0.1922 - val_mae: 0.4356
Epoch 43/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0278 - mae: 0.1045 - val_loss: 0.1825 - val_mae: 0.4245
Epoch 44/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0248 - mae: 0.0982 - val_loss: 0.1739 - val_mae: 0.4143
Epoch 45/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0248 - mae: 0.0972 - val_loss: 0.1652 - val_mae: 0.4038
Epoch 46/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0232 - mae: 0.0938 - val_loss: 0.1568 - val_mae: 0.3933
Epoch 47/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0225 - mae: 0.0921 - val_loss: 0.1492 - val_mae: 0.3835
Epoch 48/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0212 - mae: 0.0907 - val_loss: 0.1417 - val_mae: 0.3738
Epoch 49/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0185 - mae: 0.0860 - val_loss: 0.1346 - val_mae: 0.3642
Epoch 50/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0190 - mae: 0.0847 - val_loss: 0.1274 - val_mae: 0.3541
Epoch 51/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0184 - mae: 0.0847 - val_loss: 0.1195 - val_mae: 0.3429
Epoch 52/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0184 - mae: 0.0842 - val_loss: 0.1118 - val_mae: 0.3316
Epoch 53/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0164 - mae: 0.0793 - val_loss: 0.1052 - val_mae: 0.3215
Epoch 54/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0161 - mae: 0.0782 - val_loss: 0.0985 - val_mae: 0.3111
Epoch 55/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0139 - mae: 0.0737 - val_loss: 0.0929 - val_mae: 0.3019
Epoch 56/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0144 - mae: 0.0749 - val_loss: 0.0870 - val_mae: 0.2922
Epoch 57/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0137 - mae: 0.0734 - val_loss: 0.0814 - val_mae: 0.2825
Epoch 58/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0123 - mae: 0.0704 - val_loss: 0.0762 - val_mae: 0.2732
Epoch 59/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0121 - mae: 0.0699 - val_loss: 0.0714 - val_mae: 0.2644
Epoch 60/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0125 - mae: 0.0710 - val_loss: 0.0665 - val_mae: 0.2549
Epoch 61/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0126 - mae: 0.0700 - val_loss: 0.0620 - val_mae: 0.2460
Epoch 62/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0105 - mae: 0.0652 - val_loss: 0.0579 - val_mae: 0.2377
Epoch 63/500
40/40 [==============================] - 0s 7ms/step - loss: 0.0116 - mae: 0.0675 - val_loss: 0.0537 - val_mae: 0.2289
Epoch 64/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0107 - mae: 0.0652 - val_loss: 0.0498 - val_mae: 0.2203
Epoch 65/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0101 - mae: 0.0643 - val_loss: 0.0461 - val_mae: 0.2118
Epoch 66/500
40/40 [==============================] - 0s 7ms/step - loss: 0.0105 - mae: 0.0657 - val_loss: 0.0430 - val_mae: 0.2043
Epoch 67/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0102 - mae: 0.0647 - val_loss: 0.0397 - val_mae: 0.1962
Epoch 68/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0087 - mae: 0.0604 - val_loss: 0.0368 - val_mae: 0.1889
Epoch 69/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0094 - mae: 0.0621 - val_loss: 0.0340 - val_mae: 0.1813
Epoch 70/500
40/40 [==============================] - 0s 7ms/step - loss: 0.0085 - mae: 0.0605 - val_loss: 0.0312 - val_mae: 0.1735
Epoch 71/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0075 - mae: 0.0565 - val_loss: 0.0288 - val_mae: 0.1664
Epoch 72/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0078 - mae: 0.0570 - val_loss: 0.0267 - val_mae: 0.1601
Epoch 73/500
40/40 [==============================] - 0s 7ms/step - loss: 0.0078 - mae: 0.0563 - val_loss: 0.0249 - val_mae: 0.1545
Epoch 74/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0089 - mae: 0.0596 - val_loss: 0.0230 - val_mae: 0.1482
Epoch 75/500
40/40 [==============================] - 0s 12ms/step - loss: 0.0072 - mae: 0.0552 - val_loss: 0.0214 - val_mae: 0.1429
Epoch 76/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0073 - mae: 0.0542 - val_loss: 0.0199 - val_mae: 0.1375
Epoch 77/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0081 - mae: 0.0570 - val_loss: 0.0184 - val_mae: 0.1322
Epoch 78/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0075 - mae: 0.0542 - val_loss: 0.0172 - val_mae: 0.1274
Epoch 79/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0069 - mae: 0.0533 - val_loss: 0.0161 - val_mae: 0.1231
Epoch 80/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0066 - mae: 0.0514 - val_loss: 0.0151 - val_mae: 0.1190
Epoch 81/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0083 - mae: 0.0556 - val_loss: 0.0138 - val_mae: 0.1135
Epoch 82/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0066 - mae: 0.0519 - val_loss: 0.0132 - val_mae: 0.1109
Epoch 83/500
40/40 [==============================] - 0s 9ms/step - loss: 0.0063 - mae: 0.0514 - val_loss: 0.0125 - val_mae: 0.1080
Epoch 84/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0067 - mae: 0.0521 - val_loss: 0.0119 - val_mae: 0.1050
Epoch 85/500
40/40 [==============================] - 0s 11ms/step - loss: 0.0063 - mae: 0.0503 - val_loss: 0.0113 - val_mae: 0.1021
Epoch 86/500
40/40 [==============================] - 0s 11ms/step - loss: 0.0070 - mae: 0.0534 - val_loss: 0.0106 - val_mae: 0.0989
Epoch 87/500
40/40 [==============================] - 0s 10ms/step - loss: 0.0069 - mae: 0.0530 - val_loss: 0.0102 - val_mae: 0.0967
Epoch 88/500
40/40 [==============================] - 0s 11ms/step - loss: 0.0065 - mae: 0.0513 - val_loss: 0.0098 - val_mae: 0.0947
Epoch 89/500
40/40 [==============================] - 0s 12ms/step - loss: 0.0060 - mae: 0.0503 - val_loss: 0.0091 - val_mae: 0.0913
Epoch 90/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0061 - mae: 0.0501 - val_loss: 0.0086 - val_mae: 0.0884
Epoch 91/500
40/40 [==============================] - 0s 9ms/step - loss: 0.0057 - mae: 0.0484 - val_loss: 0.0080 - val_mae: 0.0847
Epoch 92/500
40/40 [==============================] - 0s 12ms/step - loss: 0.0067 - mae: 0.0513 - val_loss: 0.0075 - val_mae: 0.0822
Epoch 93/500
40/40 [==============================] - 0s 11ms/step - loss: 0.0066 - mae: 0.0517 - val_loss: 0.0075 - val_mae: 0.0821
Epoch 94/500
40/40 [==============================] - 1s 14ms/step - loss: 0.0060 - mae: 0.0498 - val_loss: 0.0071 - val_mae: 0.0798
Epoch 95/500
40/40 [==============================] - 0s 9ms/step - loss: 0.0064 - mae: 0.0514 - val_loss: 0.0068 - val_mae: 0.0775
Epoch 96/500
40/40 [==============================] - 0s 7ms/step - loss: 0.0055 - mae: 0.0487 - val_loss: 0.0067 - val_mae: 0.0771
Epoch 97/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0057 - mae: 0.0485 - val_loss: 0.0065 - val_mae: 0.0759
Epoch 98/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0060 - mae: 0.0500 - val_loss: 0.0061 - val_mae: 0.0733
Epoch 99/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0061 - mae: 0.0507 - val_loss: 0.0060 - val_mae: 0.0728
Epoch 100/500
40/40 [==============================] - 0s 9ms/step - loss: 0.0054 - mae: 0.0475 - val_loss: 0.0058 - val_mae: 0.0715
Epoch 101/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0053 - mae: 0.0467 - val_loss: 0.0056 - val_mae: 0.0701
Epoch 102/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0056 - mae: 0.0475 - val_loss: 0.0055 - val_mae: 0.0693
Epoch 103/500
40/40 [==============================] - 0s 12ms/step - loss: 0.0052 - mae: 0.0477 - val_loss: 0.0055 - val_mae: 0.0690
Epoch 104/500
40/40 [==============================] - 0s 9ms/step - loss: 0.0054 - mae: 0.0473 - val_loss: 0.0056 - val_mae: 0.0696
Epoch 105/500
40/40 [==============================] - 0s 10ms/step - loss: 0.0062 - mae: 0.0491 - val_loss: 0.0054 - val_mae: 0.0680
Epoch 106/500
40/40 [==============================] - 0s 11ms/step - loss: 0.0059 - mae: 0.0492 - val_loss: 0.0049 - val_mae: 0.0646
Epoch 107/500
40/40 [==============================] - 0s 10ms/step - loss: 0.0052 - mae: 0.0474 - val_loss: 0.0046 - val_mae: 0.0622
Epoch 108/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0054 - mae: 0.0479 - val_loss: 0.0042 - val_mae: 0.0595
Epoch 109/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0062 - mae: 0.0491 - val_loss: 0.0037 - val_mae: 0.0553
Epoch 110/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0052 - mae: 0.0478 - val_loss: 0.0040 - val_mae: 0.0574
Epoch 111/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0054 - mae: 0.0473 - val_loss: 0.0040 - val_mae: 0.0581
Epoch 112/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0061 - mae: 0.0486 - val_loss: 0.0038 - val_mae: 0.0562
Epoch 113/500
40/40 [==============================] - 0s 10ms/step - loss: 0.0054 - mae: 0.0471 - val_loss: 0.0039 - val_mae: 0.0567
Epoch 114/500
40/40 [==============================] - 0s 11ms/step - loss: 0.0057 - mae: 0.0479 - val_loss: 0.0035 - val_mae: 0.0534
Epoch 115/500
40/40 [==============================] - 0s 7ms/step - loss: 0.0054 - mae: 0.0467 - val_loss: 0.0033 - val_mae: 0.0518
Epoch 116/500
40/40 [==============================] - 0s 7ms/step - loss: 0.0053 - mae: 0.0458 - val_loss: 0.0032 - val_mae: 0.0508
Epoch 117/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0054 - mae: 0.0461 - val_loss: 0.0037 - val_mae: 0.0549
Epoch 118/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0053 - mae: 0.0463 - val_loss: 0.0036 - val_mae: 0.0546
Epoch 119/500
40/40 [==============================] - 0s 9ms/step - loss: 0.0051 - mae: 0.0446 - val_loss: 0.0035 - val_mae: 0.0538
Epoch 120/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0051 - mae: 0.0457 - val_loss: 0.0034 - val_mae: 0.0530
Epoch 121/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0046 - mae: 0.0436 - val_loss: 0.0035 - val_mae: 0.0534
Epoch 122/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0053 - mae: 0.0458 - val_loss: 0.0034 - val_mae: 0.0526
Epoch 123/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0052 - mae: 0.0446 - val_loss: 0.0033 - val_mae: 0.0521
Epoch 124/500
40/40 [==============================] - 0s 7ms/step - loss: 0.0058 - mae: 0.0463 - val_loss: 0.0033 - val_mae: 0.0521
Epoch 125/500
40/40 [==============================] - 0s 7ms/step - loss: 0.0049 - mae: 0.0458 - val_loss: 0.0032 - val_mae: 0.0511
Epoch 126/500
40/40 [==============================] - 0s 10ms/step - loss: 0.0052 - mae: 0.0452 - val_loss: 0.0029 - val_mae: 0.0480
Epoch 127/500
40/40 [==============================] - 0s 11ms/step - loss: 0.0053 - mae: 0.0459 - val_loss: 0.0030 - val_mae: 0.0492
Epoch 128/500
40/40 [==============================] - 0s 12ms/step - loss: 0.0051 - mae: 0.0453 - val_loss: 0.0029 - val_mae: 0.0480
Epoch 129/500
40/40 [==============================] - 1s 17ms/step - loss: 0.0051 - mae: 0.0453 - val_loss: 0.0030 - val_mae: 0.0490
Epoch 130/500
40/40 [==============================] - 0s 12ms/step - loss: 0.0049 - mae: 0.0443 - val_loss: 0.0031 - val_mae: 0.0500
Epoch 131/500
40/40 [==============================] - 0s 9ms/step - loss: 0.0050 - mae: 0.0453 - val_loss: 0.0027 - val_mae: 0.0466
Epoch 132/500
40/40 [==============================] - 0s 7ms/step - loss: 0.0049 - mae: 0.0440 - val_loss: 0.0025 - val_mae: 0.0438
Epoch 133/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0048 - mae: 0.0436 - val_loss: 0.0024 - val_mae: 0.0431
Epoch 134/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0052 - mae: 0.0450 - val_loss: 0.0026 - val_mae: 0.0449
Epoch 135/500
40/40 [==============================] - 0s 10ms/step - loss: 0.0044 - mae: 0.0431 - val_loss: 0.0026 - val_mae: 0.0451
Epoch 136/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0042 - mae: 0.0413 - val_loss: 0.0027 - val_mae: 0.0460
Epoch 137/500
40/40 [==============================] - 0s 12ms/step - loss: 0.0048 - mae: 0.0435 - val_loss: 0.0027 - val_mae: 0.0464
Epoch 138/500
40/40 [==============================] - 0s 11ms/step - loss: 0.0043 - mae: 0.0420 - val_loss: 0.0026 - val_mae: 0.0451
Epoch 139/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0049 - mae: 0.0438 - val_loss: 0.0024 - val_mae: 0.0427
Epoch 140/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0047 - mae: 0.0435 - val_loss: 0.0021 - val_mae: 0.0396
Epoch 141/500
40/40 [==============================] - 0s 12ms/step - loss: 0.0043 - mae: 0.0413 - val_loss: 0.0023 - val_mae: 0.0420
Epoch 142/500
40/40 [==============================] - 0s 9ms/step - loss: 0.0041 - mae: 0.0409 - val_loss: 0.0022 - val_mae: 0.0411
Epoch 143/500
40/40 [==============================] - 1s 14ms/step - loss: 0.0051 - mae: 0.0435 - val_loss: 0.0022 - val_mae: 0.0406
Epoch 144/500
40/40 [==============================] - 0s 7ms/step - loss: 0.0048 - mae: 0.0432 - val_loss: 0.0023 - val_mae: 0.0420
Epoch 145/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0045 - mae: 0.0427 - val_loss: 0.0021 - val_mae: 0.0391
Epoch 146/500
40/40 [==============================] - 0s 11ms/step - loss: 0.0043 - mae: 0.0418 - val_loss: 0.0020 - val_mae: 0.0383
Epoch 147/500
40/40 [==============================] - 0s 7ms/step - loss: 0.0049 - mae: 0.0431 - val_loss: 0.0021 - val_mae: 0.0391
Epoch 148/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0046 - mae: 0.0425 - val_loss: 0.0020 - val_mae: 0.0385
Epoch 149/500
40/40 [==============================] - 0s 7ms/step - loss: 0.0050 - mae: 0.0432 - val_loss: 0.0020 - val_mae: 0.0379
Epoch 150/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0048 - mae: 0.0423 - val_loss: 0.0019 - val_mae: 0.0367
Epoch 151/500
40/40 [==============================] - 0s 9ms/step - loss: 0.0042 - mae: 0.0406 - val_loss: 0.0020 - val_mae: 0.0386
Epoch 152/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0046 - mae: 0.0420 - val_loss: 0.0020 - val_mae: 0.0388
Epoch 153/500
40/40 [==============================] - 0s 7ms/step - loss: 0.0040 - mae: 0.0392 - val_loss: 0.0019 - val_mae: 0.0375
Epoch 154/500
40/40 [==============================] - 0s 8ms/step - loss: 0.0039 - mae: 0.0395 - val_loss: 0.0019 - val_mae: 0.0365
Epoch 155/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0047 - mae: 0.0420 - val_loss: 0.0017 - val_mae: 0.0351
Epoch 156/500
40/40 [==============================] - 0s 9ms/step - loss: 0.0044 - mae: 0.0406 - val_loss: 0.0017 - val_mae: 0.0344
Epoch 157/500
40/40 [==============================] - 0s 11ms/step - loss: 0.0039 - mae: 0.0397 - val_loss: 0.0017 - val_mae: 0.0353
Epoch 158/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0040 - mae: 0.0398 - val_loss: 0.0017 - val_mae: 0.0348
Epoch 159/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0040 - mae: 0.0398 - val_loss: 0.0018 - val_mae: 0.0354
Epoch 160/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0038 - mae: 0.0399 - val_loss: 0.0017 - val_mae: 0.0342
Epoch 161/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0041 - mae: 0.0402 - val_loss: 0.0014 - val_mae: 0.0312
Epoch 162/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0040 - mae: 0.0405 - val_loss: 0.0014 - val_mae: 0.0312
Epoch 163/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0043 - mae: 0.0400 - val_loss: 0.0015 - val_mae: 0.0323
Epoch 164/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0043 - mae: 0.0401 - val_loss: 0.0014 - val_mae: 0.0308
Epoch 165/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0044 - mae: 0.0399 - val_loss: 0.0015 - val_mae: 0.0322
Epoch 166/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0038 - mae: 0.0379 - val_loss: 0.0014 - val_mae: 0.0304
Epoch 167/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0040 - mae: 0.0391 - val_loss: 0.0014 - val_mae: 0.0312
Epoch 168/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0040 - mae: 0.0389 - val_loss: 0.0015 - val_mae: 0.0319
Epoch 169/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0046 - mae: 0.0404 - val_loss: 0.0016 - val_mae: 0.0332
Epoch 170/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0041 - mae: 0.0394 - val_loss: 0.0015 - val_mae: 0.0325
Epoch 171/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0044 - mae: 0.0398 - val_loss: 0.0014 - val_mae: 0.0307
Epoch 172/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0041 - mae: 0.0394 - val_loss: 0.0013 - val_mae: 0.0290
Epoch 173/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0043 - mae: 0.0398 - val_loss: 0.0013 - val_mae: 0.0291
Epoch 174/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0039 - mae: 0.0386 - val_loss: 0.0014 - val_mae: 0.0302
Epoch 175/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0040 - mae: 0.0391 - val_loss: 0.0015 - val_mae: 0.0318
Epoch 176/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0036 - mae: 0.0380 - val_loss: 0.0015 - val_mae: 0.0318
Epoch 177/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0036 - mae: 0.0381 - val_loss: 0.0015 - val_mae: 0.0326
Epoch 178/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0041 - mae: 0.0392 - val_loss: 0.0015 - val_mae: 0.0321
Epoch 179/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0038 - mae: 0.0382 - val_loss: 0.0013 - val_mae: 0.0291
Epoch 180/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0039 - mae: 0.0383 - val_loss: 0.0014 - val_mae: 0.0311
Epoch 181/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0038 - mae: 0.0382 - val_loss: 0.0016 - val_mae: 0.0333
Epoch 182/500
30/40 [=====================>........] - ETA: 0s - loss: 0.0034 - mae: 0.0377Restoring model weights from the end of the best epoch: 172.
40/40 [==============================] - 0s 3ms/step - loss: 0.0036 - mae: 0.0382 - val_loss: 0.0016 - val_mae: 0.0340
Epoch 182: early stopping
13/13 [==============================] - 0s 2ms/step - loss: 0.0022 - mae: 0.0417
Test Loss: 0.0021916949190199375
Test MAE: 0.041657596826553345
13/13 [==============================] - 0s 2ms/step

Predicting Returns¶

Next, let us try the model on the daily returns data which is stationary.

target_variable = 'Returns'
train_data, test_data = train_test_data(target_variable)
mlp(target_variable, train_data, test_data)

Model: "sequential_2"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_6 (Dense)             (None, 64)                384       
                                                                 
 dropout_4 (Dropout)         (None, 64)                0         
                                                                 
 dense_7 (Dense)             (None, 64)                4160      
                                                                 
 dropout_5 (Dropout)         (None, 64)                0         
                                                                 
 dense_8 (Dense)             (None, 1)                 65        
                                                                 
=================================================================
Total params: 4609 (18.00 KB)
Trainable params: 4609 (18.00 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

<ipython-input-35-db7a072d7ce2>:13: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  df_copy[f'lag_{i}'] = df_copy[target_variable].shift(i)

Epoch 1/500
40/40 [==============================] - 1s 9ms/step - loss: 0.3398 - mae: 0.5715 - val_loss: 0.3250 - val_mae: 0.5543
Epoch 2/500
40/40 [==============================] - 0s 6ms/step - loss: 0.3170 - mae: 0.5515 - val_loss: 0.3029 - val_mae: 0.5341
Epoch 3/500
40/40 [==============================] - 0s 5ms/step - loss: 0.2968 - mae: 0.5328 - val_loss: 0.2819 - val_mae: 0.5141
Epoch 4/500
40/40 [==============================] - 0s 4ms/step - loss: 0.2739 - mae: 0.5105 - val_loss: 0.2621 - val_mae: 0.4945
Epoch 5/500
40/40 [==============================] - 0s 5ms/step - loss: 0.2505 - mae: 0.4869 - val_loss: 0.2433 - val_mae: 0.4752
Epoch 6/500
40/40 [==============================] - 0s 4ms/step - loss: 0.2402 - mae: 0.4768 - val_loss: 0.2253 - val_mae: 0.4559
Epoch 7/500
40/40 [==============================] - 0s 5ms/step - loss: 0.2221 - mae: 0.4581 - val_loss: 0.2084 - val_mae: 0.4370
Epoch 8/500
40/40 [==============================] - 0s 4ms/step - loss: 0.2049 - mae: 0.4376 - val_loss: 0.1925 - val_mae: 0.4185
Epoch 9/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1901 - mae: 0.4194 - val_loss: 0.1776 - val_mae: 0.4004
Epoch 10/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1737 - mae: 0.4013 - val_loss: 0.1635 - val_mae: 0.3828
Epoch 11/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1628 - mae: 0.3867 - val_loss: 0.1503 - val_mae: 0.3655
Epoch 12/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1455 - mae: 0.3635 - val_loss: 0.1380 - val_mae: 0.3486
Epoch 13/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1370 - mae: 0.3521 - val_loss: 0.1265 - val_mae: 0.3322
Epoch 14/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1259 - mae: 0.3350 - val_loss: 0.1158 - val_mae: 0.3160
Epoch 15/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1130 - mae: 0.3142 - val_loss: 0.1059 - val_mae: 0.3006
Epoch 16/500
40/40 [==============================] - 0s 3ms/step - loss: 0.1017 - mae: 0.2959 - val_loss: 0.0969 - val_mae: 0.2859
Epoch 17/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0961 - mae: 0.2864 - val_loss: 0.0884 - val_mae: 0.2713
Epoch 18/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0861 - mae: 0.2702 - val_loss: 0.0804 - val_mae: 0.2569
Epoch 19/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0797 - mae: 0.2568 - val_loss: 0.0731 - val_mae: 0.2429
Epoch 20/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0727 - mae: 0.2414 - val_loss: 0.0665 - val_mae: 0.2298
Epoch 21/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0655 - mae: 0.2257 - val_loss: 0.0604 - val_mae: 0.2170
Epoch 22/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0580 - mae: 0.2122 - val_loss: 0.0550 - val_mae: 0.2050
Epoch 23/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0529 - mae: 0.2011 - val_loss: 0.0499 - val_mae: 0.1934
Epoch 24/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0497 - mae: 0.1932 - val_loss: 0.0455 - val_mae: 0.1828
Epoch 25/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0456 - mae: 0.1804 - val_loss: 0.0415 - val_mae: 0.1729
Epoch 26/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0424 - mae: 0.1721 - val_loss: 0.0381 - val_mae: 0.1642
Epoch 27/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0379 - mae: 0.1602 - val_loss: 0.0350 - val_mae: 0.1560
Epoch 28/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0360 - mae: 0.1557 - val_loss: 0.0324 - val_mae: 0.1489
Epoch 29/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0328 - mae: 0.1472 - val_loss: 0.0302 - val_mae: 0.1427
Epoch 30/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0305 - mae: 0.1400 - val_loss: 0.0283 - val_mae: 0.1375
Epoch 31/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0283 - mae: 0.1343 - val_loss: 0.0267 - val_mae: 0.1328
Epoch 32/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0283 - mae: 0.1348 - val_loss: 0.0254 - val_mae: 0.1289
Epoch 33/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0267 - mae: 0.1295 - val_loss: 0.0242 - val_mae: 0.1252
Epoch 34/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0267 - mae: 0.1305 - val_loss: 0.0232 - val_mae: 0.1219
Epoch 35/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0257 - mae: 0.1260 - val_loss: 0.0225 - val_mae: 0.1192
Epoch 36/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0251 - mae: 0.1254 - val_loss: 0.0218 - val_mae: 0.1167
Epoch 37/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0232 - mae: 0.1207 - val_loss: 0.0213 - val_mae: 0.1147
Epoch 38/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0220 - mae: 0.1180 - val_loss: 0.0209 - val_mae: 0.1131
Epoch 39/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0230 - mae: 0.1206 - val_loss: 0.0205 - val_mae: 0.1117
Epoch 40/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0217 - mae: 0.1147 - val_loss: 0.0202 - val_mae: 0.1107
Epoch 41/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0221 - mae: 0.1173 - val_loss: 0.0201 - val_mae: 0.1101
Epoch 42/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0216 - mae: 0.1172 - val_loss: 0.0199 - val_mae: 0.1096
Epoch 43/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0227 - mae: 0.1191 - val_loss: 0.0198 - val_mae: 0.1092
Epoch 44/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0206 - mae: 0.1150 - val_loss: 0.0197 - val_mae: 0.1088
Epoch 45/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0215 - mae: 0.1143 - val_loss: 0.0197 - val_mae: 0.1086
Epoch 46/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0214 - mae: 0.1143 - val_loss: 0.0196 - val_mae: 0.1084
Epoch 47/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0213 - mae: 0.1152 - val_loss: 0.0196 - val_mae: 0.1083
Epoch 48/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0197 - mae: 0.1111 - val_loss: 0.0196 - val_mae: 0.1081
Epoch 49/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0224 - mae: 0.1186 - val_loss: 0.0195 - val_mae: 0.1079
Epoch 50/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0202 - mae: 0.1126 - val_loss: 0.0195 - val_mae: 0.1078
Epoch 51/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0217 - mae: 0.1159 - val_loss: 0.0195 - val_mae: 0.1077
Epoch 52/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0211 - mae: 0.1156 - val_loss: 0.0195 - val_mae: 0.1076
Epoch 53/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0208 - mae: 0.1125 - val_loss: 0.0195 - val_mae: 0.1076
Epoch 54/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0205 - mae: 0.1126 - val_loss: 0.0195 - val_mae: 0.1076
Epoch 55/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0196 - mae: 0.1102 - val_loss: 0.0195 - val_mae: 0.1075
Epoch 56/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0204 - mae: 0.1122 - val_loss: 0.0195 - val_mae: 0.1076
Epoch 57/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0191 - mae: 0.1086 - val_loss: 0.0194 - val_mae: 0.1075
Epoch 58/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0199 - mae: 0.1104 - val_loss: 0.0194 - val_mae: 0.1074
Epoch 59/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0202 - mae: 0.1119 - val_loss: 0.0194 - val_mae: 0.1074
Epoch 60/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0197 - mae: 0.1114 - val_loss: 0.0194 - val_mae: 0.1074
Epoch 61/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0188 - mae: 0.1077 - val_loss: 0.0194 - val_mae: 0.1074
Epoch 62/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0198 - mae: 0.1097 - val_loss: 0.0194 - val_mae: 0.1074
Epoch 63/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0192 - mae: 0.1089 - val_loss: 0.0194 - val_mae: 0.1073
Epoch 64/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0191 - mae: 0.1086 - val_loss: 0.0194 - val_mae: 0.1074
Epoch 65/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0192 - mae: 0.1086 - val_loss: 0.0194 - val_mae: 0.1073
Epoch 66/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0196 - mae: 0.1092 - val_loss: 0.0194 - val_mae: 0.1073
Epoch 67/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0190 - mae: 0.1076 - val_loss: 0.0194 - val_mae: 0.1073
Epoch 68/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0187 - mae: 0.1077 - val_loss: 0.0194 - val_mae: 0.1074
Epoch 69/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0201 - mae: 0.1126 - val_loss: 0.0194 - val_mae: 0.1074
Epoch 70/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0197 - mae: 0.1118 - val_loss: 0.0194 - val_mae: 0.1073
Epoch 71/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0191 - mae: 0.1078 - val_loss: 0.0194 - val_mae: 0.1072
Epoch 72/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0188 - mae: 0.1072 - val_loss: 0.0193 - val_mae: 0.1071
Epoch 73/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0181 - mae: 0.1060 - val_loss: 0.0193 - val_mae: 0.1071
Epoch 74/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0210 - mae: 0.1137 - val_loss: 0.0193 - val_mae: 0.1071
Epoch 75/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0181 - mae: 0.1050 - val_loss: 0.0193 - val_mae: 0.1071
Epoch 76/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0191 - mae: 0.1074 - val_loss: 0.0193 - val_mae: 0.1070
Epoch 77/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0178 - mae: 0.1052 - val_loss: 0.0193 - val_mae: 0.1069
Epoch 78/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0201 - mae: 0.1124 - val_loss: 0.0193 - val_mae: 0.1069
Epoch 79/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0184 - mae: 0.1071 - val_loss: 0.0193 - val_mae: 0.1068
Epoch 80/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0191 - mae: 0.1090 - val_loss: 0.0193 - val_mae: 0.1068
Epoch 81/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0193 - mae: 0.1097 - val_loss: 0.0193 - val_mae: 0.1069
Epoch 82/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0179 - mae: 0.1056 - val_loss: 0.0193 - val_mae: 0.1069
Epoch 83/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0191 - mae: 0.1083 - val_loss: 0.0193 - val_mae: 0.1068
Epoch 84/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0180 - mae: 0.1057 - val_loss: 0.0193 - val_mae: 0.1068
Epoch 85/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0188 - mae: 0.1063 - val_loss: 0.0193 - val_mae: 0.1068
Epoch 86/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0173 - mae: 0.1031 - val_loss: 0.0193 - val_mae: 0.1068
Epoch 87/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0178 - mae: 0.1033 - val_loss: 0.0193 - val_mae: 0.1068
Epoch 88/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0176 - mae: 0.1033 - val_loss: 0.0193 - val_mae: 0.1068
Epoch 89/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0177 - mae: 0.1038 - val_loss: 0.0193 - val_mae: 0.1068
Epoch 90/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0178 - mae: 0.1028 - val_loss: 0.0193 - val_mae: 0.1068
Epoch 91/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0172 - mae: 0.1021 - val_loss: 0.0193 - val_mae: 0.1069
Epoch 92/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0178 - mae: 0.1033 - val_loss: 0.0193 - val_mae: 0.1069
Epoch 93/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0174 - mae: 0.1023 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 94/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0179 - mae: 0.1040 - val_loss: 0.0192 - val_mae: 0.1067
Epoch 95/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0182 - mae: 0.1047 - val_loss: 0.0192 - val_mae: 0.1067
Epoch 96/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0179 - mae: 0.1054 - val_loss: 0.0192 - val_mae: 0.1067
Epoch 97/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0172 - mae: 0.1031 - val_loss: 0.0192 - val_mae: 0.1067
Epoch 98/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0176 - mae: 0.1032 - val_loss: 0.0192 - val_mae: 0.1067
Epoch 99/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0165 - mae: 0.1003 - val_loss: 0.0192 - val_mae: 0.1067
Epoch 100/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0164 - mae: 0.0988 - val_loss: 0.0192 - val_mae: 0.1067
Epoch 101/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0165 - mae: 0.1002 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 102/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0164 - mae: 0.0990 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 103/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0176 - mae: 0.1021 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 104/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0173 - mae: 0.1026 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 105/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0161 - mae: 0.0981 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 106/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0158 - mae: 0.0983 - val_loss: 0.0192 - val_mae: 0.1067
Epoch 107/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0159 - mae: 0.0991 - val_loss: 0.0192 - val_mae: 0.1067
Epoch 108/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0154 - mae: 0.0959 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 109/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0159 - mae: 0.0982 - val_loss: 0.0192 - val_mae: 0.1067
Epoch 110/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0165 - mae: 0.0995 - val_loss: 0.0192 - val_mae: 0.1067
Epoch 111/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0157 - mae: 0.0972 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 112/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0162 - mae: 0.0978 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 113/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0146 - mae: 0.0947 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 114/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0158 - mae: 0.0979 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 115/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0148 - mae: 0.0943 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 116/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0159 - mae: 0.0992 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 117/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0161 - mae: 0.0982 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 118/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0165 - mae: 0.1016 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 119/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0142 - mae: 0.0910 - val_loss: 0.0192 - val_mae: 0.1069
Epoch 120/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0160 - mae: 0.0974 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 121/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0162 - mae: 0.0986 - val_loss: 0.0192 - val_mae: 0.1069
Epoch 122/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0149 - mae: 0.0954 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 123/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0150 - mae: 0.0940 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 124/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0153 - mae: 0.0951 - val_loss: 0.0192 - val_mae: 0.1069
Epoch 125/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0156 - mae: 0.0957 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 126/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0146 - mae: 0.0947 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 127/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0151 - mae: 0.0947 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 128/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0152 - mae: 0.0954 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 129/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0146 - mae: 0.0937 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 130/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0144 - mae: 0.0912 - val_loss: 0.0192 - val_mae: 0.1068
Epoch 131/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0149 - mae: 0.0939 - val_loss: 0.0192 - val_mae: 0.1069
Epoch 132/500
30/40 [=====================>........] - ETA: 0s - loss: 0.0151 - mae: 0.0937Restoring model weights from the end of the best epoch: 122.
40/40 [==============================] - 0s 3ms/step - loss: 0.0147 - mae: 0.0932 - val_loss: 0.0192 - val_mae: 0.1069
Epoch 132: early stopping
13/13 [==============================] - 0s 2ms/step - loss: 0.0140 - mae: 0.0884
Test Loss: 0.01403221394866705
Test MAE: 0.08835497498512268
13/13 [==============================] - 0s 2ms/step

Predicting Fractional differences¶

Finally, let's see if model does better on frational differences.

target_variable = 'Fractional'
train_data, test_data = train_test_data(target_variable)
mlp(target_variable, train_data, test_data)

Model: "sequential_3"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_9 (Dense)             (None, 64)                384       
                                                                 
 dropout_6 (Dropout)         (None, 64)                0         
                                                                 
 dense_10 (Dense)            (None, 64)                4160      
                                                                 
 dropout_7 (Dropout)         (None, 64)                0         
                                                                 
 dense_11 (Dense)            (None, 1)                 65        
                                                                 
=================================================================
Total params: 4609 (18.00 KB)
Trainable params: 4609 (18.00 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

<ipython-input-35-db7a072d7ce2>:13: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  df_copy[f'lag_{i}'] = df_copy[target_variable].shift(i)

Epoch 1/500
40/40 [==============================] - 1s 7ms/step - loss: 0.1055 - mae: 0.3063 - val_loss: 0.1275 - val_mae: 0.3316
Epoch 2/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0948 - mae: 0.2888 - val_loss: 0.1174 - val_mae: 0.3163
Epoch 3/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0891 - mae: 0.2776 - val_loss: 0.1079 - val_mae: 0.3013
Epoch 4/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0817 - mae: 0.2653 - val_loss: 0.0991 - val_mae: 0.2866
Epoch 5/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0764 - mae: 0.2538 - val_loss: 0.0909 - val_mae: 0.2725
Epoch 6/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0685 - mae: 0.2384 - val_loss: 0.0834 - val_mae: 0.2591
Epoch 7/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0609 - mae: 0.2222 - val_loss: 0.0765 - val_mae: 0.2462
Epoch 8/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0577 - mae: 0.2141 - val_loss: 0.0702 - val_mae: 0.2339
Epoch 9/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0511 - mae: 0.1989 - val_loss: 0.0645 - val_mae: 0.2222
Epoch 10/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0494 - mae: 0.1925 - val_loss: 0.0592 - val_mae: 0.2110
Epoch 11/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0434 - mae: 0.1783 - val_loss: 0.0544 - val_mae: 0.2004
Epoch 12/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0430 - mae: 0.1740 - val_loss: 0.0500 - val_mae: 0.1904
Epoch 13/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0376 - mae: 0.1633 - val_loss: 0.0461 - val_mae: 0.1811
Epoch 14/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0347 - mae: 0.1518 - val_loss: 0.0427 - val_mae: 0.1728
Epoch 15/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0320 - mae: 0.1459 - val_loss: 0.0397 - val_mae: 0.1653
Epoch 16/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0290 - mae: 0.1381 - val_loss: 0.0371 - val_mae: 0.1588
Epoch 17/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0279 - mae: 0.1335 - val_loss: 0.0346 - val_mae: 0.1524
Epoch 18/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0258 - mae: 0.1260 - val_loss: 0.0326 - val_mae: 0.1469
Epoch 19/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0242 - mae: 0.1235 - val_loss: 0.0308 - val_mae: 0.1420
Epoch 20/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0211 - mae: 0.1137 - val_loss: 0.0293 - val_mae: 0.1378
Epoch 21/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0234 - mae: 0.1191 - val_loss: 0.0279 - val_mae: 0.1339
Epoch 22/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0201 - mae: 0.1113 - val_loss: 0.0268 - val_mae: 0.1306
Epoch 23/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0206 - mae: 0.1131 - val_loss: 0.0258 - val_mae: 0.1275
Epoch 24/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0200 - mae: 0.1109 - val_loss: 0.0251 - val_mae: 0.1250
Epoch 25/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0184 - mae: 0.1049 - val_loss: 0.0244 - val_mae: 0.1227
Epoch 26/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0191 - mae: 0.1064 - val_loss: 0.0238 - val_mae: 0.1209
Epoch 27/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0173 - mae: 0.1024 - val_loss: 0.0233 - val_mae: 0.1194
Epoch 28/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0197 - mae: 0.1086 - val_loss: 0.0230 - val_mae: 0.1184
Epoch 29/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0170 - mae: 0.1017 - val_loss: 0.0227 - val_mae: 0.1174
Epoch 30/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0173 - mae: 0.0999 - val_loss: 0.0225 - val_mae: 0.1167
Epoch 31/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0170 - mae: 0.1013 - val_loss: 0.0223 - val_mae: 0.1163
Epoch 32/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0171 - mae: 0.1018 - val_loss: 0.0221 - val_mae: 0.1158
Epoch 33/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0161 - mae: 0.0985 - val_loss: 0.0220 - val_mae: 0.1155
Epoch 34/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0165 - mae: 0.0994 - val_loss: 0.0219 - val_mae: 0.1151
Epoch 35/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0168 - mae: 0.1003 - val_loss: 0.0217 - val_mae: 0.1148
Epoch 36/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0167 - mae: 0.0986 - val_loss: 0.0217 - val_mae: 0.1146
Epoch 37/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0169 - mae: 0.1000 - val_loss: 0.0216 - val_mae: 0.1144
Epoch 38/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0157 - mae: 0.0964 - val_loss: 0.0215 - val_mae: 0.1142
Epoch 39/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0166 - mae: 0.1005 - val_loss: 0.0215 - val_mae: 0.1141
Epoch 40/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0149 - mae: 0.0947 - val_loss: 0.0215 - val_mae: 0.1141
Epoch 41/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0150 - mae: 0.0944 - val_loss: 0.0214 - val_mae: 0.1139
Epoch 42/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0157 - mae: 0.0967 - val_loss: 0.0213 - val_mae: 0.1137
Epoch 43/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0149 - mae: 0.0949 - val_loss: 0.0213 - val_mae: 0.1135
Epoch 44/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0156 - mae: 0.0960 - val_loss: 0.0213 - val_mae: 0.1135
Epoch 45/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0156 - mae: 0.0961 - val_loss: 0.0212 - val_mae: 0.1134
Epoch 46/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0159 - mae: 0.0976 - val_loss: 0.0213 - val_mae: 0.1134
Epoch 47/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0142 - mae: 0.0935 - val_loss: 0.0212 - val_mae: 0.1134
Epoch 48/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0150 - mae: 0.0960 - val_loss: 0.0212 - val_mae: 0.1133
Epoch 49/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0149 - mae: 0.0944 - val_loss: 0.0212 - val_mae: 0.1134
Epoch 50/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0143 - mae: 0.0920 - val_loss: 0.0213 - val_mae: 0.1134
Epoch 51/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0143 - mae: 0.0927 - val_loss: 0.0213 - val_mae: 0.1134
Epoch 52/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0159 - mae: 0.0972 - val_loss: 0.0213 - val_mae: 0.1133
Epoch 53/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0148 - mae: 0.0945 - val_loss: 0.0212 - val_mae: 0.1132
Epoch 54/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0147 - mae: 0.0938 - val_loss: 0.0212 - val_mae: 0.1132
Epoch 55/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0139 - mae: 0.0894 - val_loss: 0.0212 - val_mae: 0.1131
Epoch 56/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0132 - mae: 0.0874 - val_loss: 0.0212 - val_mae: 0.1132
Epoch 57/500
40/40 [==============================] - 0s 6ms/step - loss: 0.0141 - mae: 0.0927 - val_loss: 0.0212 - val_mae: 0.1131
Epoch 58/500
40/40 [==============================] - 0s 5ms/step - loss: 0.0143 - mae: 0.0915 - val_loss: 0.0212 - val_mae: 0.1132
Epoch 59/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0133 - mae: 0.0888 - val_loss: 0.0212 - val_mae: 0.1131
Epoch 60/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0136 - mae: 0.0886 - val_loss: 0.0212 - val_mae: 0.1131
Epoch 61/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0137 - mae: 0.0903 - val_loss: 0.0212 - val_mae: 0.1131
Epoch 62/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0149 - mae: 0.0932 - val_loss: 0.0212 - val_mae: 0.1131
Epoch 63/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0127 - mae: 0.0868 - val_loss: 0.0212 - val_mae: 0.1131
Epoch 64/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0137 - mae: 0.0897 - val_loss: 0.0212 - val_mae: 0.1130
Epoch 65/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0136 - mae: 0.0889 - val_loss: 0.0212 - val_mae: 0.1131
Epoch 66/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0128 - mae: 0.0868 - val_loss: 0.0213 - val_mae: 0.1132
Epoch 67/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0138 - mae: 0.0894 - val_loss: 0.0213 - val_mae: 0.1131
Epoch 68/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0140 - mae: 0.0915 - val_loss: 0.0212 - val_mae: 0.1131
Epoch 69/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0130 - mae: 0.0867 - val_loss: 0.0212 - val_mae: 0.1130
Epoch 70/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0132 - mae: 0.0876 - val_loss: 0.0212 - val_mae: 0.1130
Epoch 71/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0129 - mae: 0.0880 - val_loss: 0.0212 - val_mae: 0.1130
Epoch 72/500
40/40 [==============================] - 0s 4ms/step - loss: 0.0129 - mae: 0.0874 - val_loss: 0.0212 - val_mae: 0.1130
Epoch 73/500
40/40 [==============================] - 0s 3ms/step - loss: 0.0130 - mae: 0.0866 - val_loss: 0.0212 - val_mae: 0.1129
Epoch 74/500
29/40 [====================>.........] - ETA: 0s - loss: 0.0138 - mae: 0.0895Restoring model weights from the end of the best epoch: 64.
40/40 [==============================] - 0s 3ms/step - loss: 0.0138 - mae: 0.0893 - val_loss: 0.0212 - val_mae: 0.1130
Epoch 74: early stopping
13/13 [==============================] - 0s 3ms/step - loss: 0.0159 - mae: 0.0948
Test Loss: 0.015872713178396225
Test MAE: 0.09482176601886749
13/13 [==============================] - 0s 3ms/step

Our evaluation of the models is based on the Mean absolute error on the test set, the loss is the mean squared error. As summarized below

MLP from time series
- Test Loss: 0.00138
- Test MAE: 0.03088
MLP from stationary time series (using differentiation)
- Test Loss: 0.01418
- Test MAE: 0.0882
MLP from stationary time series (using partial differentiation)
- Test Loss: 0.0164
- Test MAE: 0.09767

In this case the model that perfomed best was the MLP from the time series, this indicates that the work done to obtain a stationary time series for autoregression might not be as important to Neural networks as it it is in the case of Autoregressive models like ARIMA or GARCH. While this is only shown for a fixed set of parameters here, however we saw the same behaviour across many different epochs and validation splits.

Convolutional Neural Network (CNN)¶

CNN are used for image analysis. The input to CNN are images. To convert the time-series data into images, we will use GAF (Gramian Angular Field) transformation on a window of 30 readings. The resulting images will be used to train the CNN.

def train_test_data_cnn(target_variable):
  """
  This method converts the time-series data into a series of images (30 x 30). These
  images are used to train CNN. The target variable here is the value of time-series
  at the window length.
  """
  Xdf = df.iloc[:,df.columns.get_loc(target_variable)]
  Xdf.dropna(inplace=True)
  X_train = []
  window_size = 30
  for i in range(window_size, Xdf.shape[0]):
    X_train.append(Xdf[i - window_size : i])
  X_train = np.array(X_train)
  #print(X_train.shape)
  # Define a "window_size"-days interval
  days = np.linspace(1, X_train.shape[0], num=X_train.shape[0])
  # Plot the overlapping time series
  #plt.figure(figsize=(6, 6))
  #for i in range(241):
  #  plt.plot(days[i : window_size + i], X_train[i, :])

  transformer = GramianAngularField()
  X_train_new = transformer.transform(X_train)
  #print(X_train_new.shape)

  # Show the results for the first time series
  #fig, ax = plt.subplots(1, 2, figsize=(10, 7), gridspec_kw={"width_ratios": [1, 1]})
  #ax[0].plot(X_train[0, :])
  #ax[1].imshow(X_train_new[0], cmap="rainbow")
  #print(X_train_new[0].shape)

  y_data = []
  for i in range(window_size, Xdf.shape[0] - window_size):
    y_data.append(Xdf[i])
  y_data = np.array(y_data)
  #y_data.shape

  X_data = X_train_new[:-window_size]
  for i in range(X_data.shape[0]):
    X_data[i] = (X_data[i] - np.min(X_data[i])) / (
        np.max(X_data[i]) - np.min(X_data[i])
    )
  # We could simply write X_data = (X_data + 1)/2
  X_data = X_data * 255

  # display several time series and the corresponding images
  #fig, ax = plt.subplots(2, 5, figsize=(20, 4))
  #for i in range(0, 125, 25):
  #  ax[0, int(i / 25)].plot(X_train[i])
  #for i in range(0, 125, 25):
  #  ax[1, int(i / 25)].imshow(X_data[i], "gray")

  # Split into train and test
  train_split = 0.8
  y_data_train = y_data[: int(train_split * y_data.shape[0])]
  y_data_test = y_data[int(train_split * y_data.shape[0]) :]
  X_data_train = X_data[: int(train_split * y_data.shape[0])]
  X_data_test = X_data[int(train_split * y_data.shape[0]) :]

  return X_data_train, X_data_test, y_data_train, y_data_test

def cnn(target_variable, X_data_train, X_data_test, y_data_train, y_data_test):
  """
  Method to train, test and plot the results of a Convolutional Neural Network (CNN) model
  for predicting the target variable.
  """
  model = get_cnn_model()
  hp_lr = 1e-5  # Learning rate
  adam = tf.keras.optimizers.Adam(learning_rate=hp_lr)  # Adam optimizer
  model.compile(optimizer=adam, loss='mean_squared_error', metrics=['mae'])

  es = EarlyStopping(
    monitor="val_loss", mode="min", verbose=1, patience=10, restore_best_weights=True
  )

  model.fit(
    x=X_data_train,
    y=y_data_train,
    epochs=500,
    batch_size=32,
    validation_data=(X_data_test, y_data_test),
    verbose=1,
    callbacks=[es],
  )
  loss, mae = model.evaluate(X_data_test, y_data_test)
  print("Test Loss:", loss)
  print("Test MAE:", mae)

  predictions = model.predict(X_data_test)

  plt.figure(figsize=(20, 6))
  plt.plot(y_data_test, label='Actual')
  plt.plot(predictions, label='Predicted')
  plt.title(f'Actual vs. Predicted Values ({target_variable})')
  plt.xlabel('Time')
  plt.ylabel('Close Price')
  plt.legend()
  plt.grid(True)
  plt.show()

Let's define a deep learning model that we'll train on the data.

def get_cnn_model():
  """
  Method to define a CNN using Tensorflow.
  """
  cnn_model = tf.keras.Sequential()
  # Input layer
  cnn_model.add(tf.keras.layers.InputLayer(input_shape=(30, 30, 1)))
  cnn_model.add(tf.keras.layers.Conv2D(16, 3, activation="relu"))
  cnn_model.add(tf.keras.layers.MaxPooling2D(2))
  cnn_model.add(tf.keras.layers.Conv2D(32, 3, activation="relu"))
  cnn_model.add(tf.keras.layers.MaxPooling2D(2))
  cnn_model.add(tf.keras.layers.Conv2D(64, 3, activation="relu"))
  cnn_model.add(tf.keras.layers.MaxPooling2D(2))
  cnn_model.add(tf.keras.layers.Flatten())
  cnn_model.add(tf.keras.layers.Dense(1024, activation="relu"))
  cnn_model.add(tf.keras.layers.Dropout(0.5, seed=1234))
  cnn_model.add(tf.keras.layers.Dense(1))
  cnn_model.summary()
  return cnn_model

Predicting Price Levels¶

Using GAF representation of the closing price levels, let us do the forecasting.

target_variable = 'Close'
X_data_train, X_data_test, y_data_train, y_data_test = train_test_data_cnn(target_variable)
cnn(target_variable, X_data_train, X_data_test, y_data_train, y_data_test)

Model: "sequential_4"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 28, 28, 16)        160       
                                                                 
 max_pooling2d (MaxPooling2  (None, 14, 14, 16)        0         
 D)                                                              
                                                                 
 conv2d_1 (Conv2D)           (None, 12, 12, 32)        4640      
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 6, 6, 32)          0         
 g2D)                                                            
                                                                 
 conv2d_2 (Conv2D)           (None, 4, 4, 64)          18496     
                                                                 
 max_pooling2d_2 (MaxPoolin  (None, 2, 2, 64)          0         
 g2D)                                                            
                                                                 
 flatten (Flatten)           (None, 256)               0         
                                                                 
 dense_12 (Dense)            (None, 1024)              263168    
                                                                 
 dropout_8 (Dropout)         (None, 1024)              0         
                                                                 
 dense_13 (Dense)            (None, 1)                 1025      
                                                                 
=================================================================
Total params: 287489 (1.10 MB)
Trainable params: 287489 (1.10 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
Epoch 1/500
49/49 [==============================] - 3s 30ms/step - loss: 8783.6934 - mae: 79.2793 - val_loss: 23919.9824 - val_mae: 153.3512
Epoch 2/500
49/49 [==============================] - 1s 24ms/step - loss: 5560.1348 - mae: 57.3450 - val_loss: 16897.6641 - val_mae: 127.9223
Epoch 3/500
49/49 [==============================] - 1s 25ms/step - loss: 3640.8965 - mae: 45.3895 - val_loss: 11647.0938 - val_mae: 104.5464
Epoch 4/500
49/49 [==============================] - 1s 24ms/step - loss: 2972.8508 - mae: 44.1704 - val_loss: 9275.5312 - val_mae: 91.9277
Epoch 5/500
49/49 [==============================] - 2s 37ms/step - loss: 2902.3555 - mae: 44.9974 - val_loss: 8593.6064 - val_mae: 88.0960
Epoch 6/500
49/49 [==============================] - 2s 40ms/step - loss: 2796.7896 - mae: 44.1494 - val_loss: 8624.0049 - val_mae: 88.4675
Epoch 7/500
49/49 [==============================] - 2s 34ms/step - loss: 2803.0557 - mae: 44.3639 - val_loss: 8441.4902 - val_mae: 87.5652
Epoch 8/500
49/49 [==============================] - 1s 24ms/step - loss: 2676.3425 - mae: 43.8440 - val_loss: 8432.2676 - val_mae: 87.6492
Epoch 9/500
49/49 [==============================] - 1s 24ms/step - loss: 2730.7681 - mae: 43.9641 - val_loss: 8380.4463 - val_mae: 87.4901
Epoch 10/500
49/49 [==============================] - 1s 24ms/step - loss: 2676.8225 - mae: 43.7876 - val_loss: 8491.7959 - val_mae: 88.2584
Epoch 11/500
49/49 [==============================] - 1s 24ms/step - loss: 2642.9917 - mae: 43.7902 - val_loss: 8252.4648 - val_mae: 86.9076
Epoch 12/500
49/49 [==============================] - 1s 24ms/step - loss: 2624.6719 - mae: 43.4977 - val_loss: 8237.3662 - val_mae: 86.9013
Epoch 13/500
49/49 [==============================] - 1s 25ms/step - loss: 2515.7878 - mae: 42.6429 - val_loss: 8255.8027 - val_mae: 87.0763
Epoch 14/500
49/49 [==============================] - 1s 25ms/step - loss: 2561.5576 - mae: 43.1373 - val_loss: 8160.2520 - val_mae: 86.5510
Epoch 15/500
49/49 [==============================] - 1s 30ms/step - loss: 2559.7727 - mae: 43.1769 - val_loss: 8079.6519 - val_mae: 86.0761
Epoch 16/500
49/49 [==============================] - 2s 40ms/step - loss: 2522.6301 - mae: 43.2910 - val_loss: 7952.2378 - val_mae: 85.3542
Epoch 17/500
49/49 [==============================] - 2s 41ms/step - loss: 2485.5303 - mae: 42.8767 - val_loss: 8168.2744 - val_mae: 86.6600
Epoch 18/500
49/49 [==============================] - 1s 25ms/step - loss: 2478.9304 - mae: 42.5799 - val_loss: 8114.6104 - val_mae: 86.3605
Epoch 19/500
49/49 [==============================] - 1s 26ms/step - loss: 2429.5735 - mae: 42.2204 - val_loss: 7747.6743 - val_mae: 84.1310
Epoch 20/500
49/49 [==============================] - 1s 24ms/step - loss: 2469.8669 - mae: 43.1291 - val_loss: 8195.1270 - val_mae: 86.8622
Epoch 21/500
49/49 [==============================] - 1s 24ms/step - loss: 2406.6201 - mae: 41.7044 - val_loss: 7970.7666 - val_mae: 85.5228
Epoch 22/500
49/49 [==============================] - 1s 24ms/step - loss: 2439.7371 - mae: 42.5585 - val_loss: 8077.5166 - val_mae: 86.1746
Epoch 23/500
49/49 [==============================] - 1s 25ms/step - loss: 2373.3589 - mae: 42.0413 - val_loss: 7941.6050 - val_mae: 85.3480
Epoch 24/500
49/49 [==============================] - 1s 23ms/step - loss: 2372.7849 - mae: 41.8654 - val_loss: 8027.4219 - val_mae: 85.8794
Epoch 25/500
49/49 [==============================] - 1s 25ms/step - loss: 2442.0447 - mae: 42.5928 - val_loss: 7854.9419 - val_mae: 84.8336
Epoch 26/500
49/49 [==============================] - 2s 36ms/step - loss: 2349.2097 - mae: 41.9530 - val_loss: 7798.0430 - val_mae: 84.4892
Epoch 27/500
49/49 [==============================] - 2s 39ms/step - loss: 2305.4712 - mae: 41.4526 - val_loss: 7750.6553 - val_mae: 84.1807
Epoch 28/500
49/49 [==============================] - 2s 33ms/step - loss: 2308.5984 - mae: 41.9087 - val_loss: 7930.1562 - val_mae: 85.2873
Epoch 29/500
48/49 [============================>.] - ETA: 0s - loss: 2326.1482 - mae: 41.6373Restoring model weights from the end of the best epoch: 19.
49/49 [==============================] - 1s 25ms/step - loss: 2329.3354 - mae: 41.6877 - val_loss: 7890.0352 - val_mae: 85.0312
Epoch 29: early stopping
13/13 [==============================] - 0s 8ms/step - loss: 7747.6743 - mae: 84.1310
Test Loss: 7747.67431640625
Test MAE: 84.13101959228516
13/13 [==============================] - 0s 7ms/step

Predicting Returns¶

Using GAF representation of the daily returns, let us do the forecasting.

target_variable = 'Returns'
X_data_train, X_data_test, y_data_train, y_data_test = train_test_data_cnn(target_variable)
cnn(target_variable, X_data_train, X_data_test, y_data_train, y_data_test)

Model: "sequential_5"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d_3 (Conv2D)           (None, 28, 28, 16)        160       
                                                                 
 max_pooling2d_3 (MaxPoolin  (None, 14, 14, 16)        0         
 g2D)                                                            
                                                                 
 conv2d_4 (Conv2D)           (None, 12, 12, 32)        4640      
                                                                 
 max_pooling2d_4 (MaxPoolin  (None, 6, 6, 32)          0         
 g2D)                                                            
                                                                 
 conv2d_5 (Conv2D)           (None, 4, 4, 64)          18496     
                                                                 
 max_pooling2d_5 (MaxPoolin  (None, 2, 2, 64)          0         
 g2D)                                                            
                                                                 
 flatten_1 (Flatten)         (None, 256)               0         
                                                                 
 dense_14 (Dense)            (None, 1024)              263168    
                                                                 
 dropout_9 (Dropout)         (None, 1024)              0         
                                                                 
 dense_15 (Dense)            (None, 1)                 1025      
                                                                 
=================================================================
Total params: 287489 (1.10 MB)
Trainable params: 287489 (1.10 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
Epoch 1/500
49/49 [==============================] - 2s 29ms/step - loss: 75.5480 - mae: 6.8638 - val_loss: 10.9320 - val_mae: 2.5514
Epoch 2/500
49/49 [==============================] - 1s 24ms/step - loss: 56.5093 - mae: 5.9214 - val_loss: 10.3993 - val_mae: 2.4862
Epoch 3/500
49/49 [==============================] - 3s 52ms/step - loss: 48.4055 - mae: 5.4574 - val_loss: 9.7827 - val_mae: 2.4141
Epoch 4/500
49/49 [==============================] - 2s 51ms/step - loss: 43.8505 - mae: 5.1768 - val_loss: 9.3888 - val_mae: 2.3516
Epoch 5/500
49/49 [==============================] - 2s 40ms/step - loss: 37.2366 - mae: 4.7942 - val_loss: 9.0869 - val_mae: 2.3030
Epoch 6/500
49/49 [==============================] - 2s 31ms/step - loss: 30.9708 - mae: 4.3831 - val_loss: 8.8371 - val_mae: 2.2627
Epoch 7/500
49/49 [==============================] - 1s 25ms/step - loss: 27.4038 - mae: 4.0886 - val_loss: 8.6142 - val_mae: 2.2312
Epoch 8/500
49/49 [==============================] - 1s 24ms/step - loss: 24.9147 - mae: 3.9293 - val_loss: 8.4405 - val_mae: 2.1974
Epoch 9/500
49/49 [==============================] - 1s 23ms/step - loss: 23.4670 - mae: 3.8285 - val_loss: 8.2655 - val_mae: 2.1747
Epoch 10/500
49/49 [==============================] - 1s 25ms/step - loss: 21.2942 - mae: 3.6127 - val_loss: 8.0363 - val_mae: 2.1432
Epoch 11/500
49/49 [==============================] - 1s 25ms/step - loss: 19.3335 - mae: 3.4614 - val_loss: 7.8773 - val_mae: 2.1165
Epoch 12/500
49/49 [==============================] - 1s 23ms/step - loss: 18.3020 - mae: 3.3630 - val_loss: 7.7569 - val_mae: 2.0975
Epoch 13/500
49/49 [==============================] - 1s 25ms/step - loss: 16.0368 - mae: 3.1727 - val_loss: 7.7116 - val_mae: 2.0903
Epoch 14/500
49/49 [==============================] - 1s 30ms/step - loss: 14.0889 - mae: 2.9430 - val_loss: 7.6066 - val_mae: 2.0780
Epoch 15/500
49/49 [==============================] - 2s 39ms/step - loss: 13.5696 - mae: 2.9256 - val_loss: 7.5026 - val_mae: 2.0637
Epoch 16/500
49/49 [==============================] - 2s 37ms/step - loss: 12.0274 - mae: 2.7201 - val_loss: 7.4836 - val_mae: 2.0651
Epoch 17/500
49/49 [==============================] - 1s 24ms/step - loss: 12.3526 - mae: 2.7285 - val_loss: 7.3903 - val_mae: 2.0482
Epoch 18/500
49/49 [==============================] - 1s 24ms/step - loss: 10.4294 - mae: 2.5194 - val_loss: 7.3544 - val_mae: 2.0419
Epoch 19/500
49/49 [==============================] - 1s 25ms/step - loss: 10.6739 - mae: 2.5494 - val_loss: 7.2518 - val_mae: 2.0264
Epoch 20/500
49/49 [==============================] - 1s 25ms/step - loss: 9.8851 - mae: 2.4489 - val_loss: 7.2197 - val_mae: 2.0200
Epoch 21/500
49/49 [==============================] - 1s 24ms/step - loss: 9.2344 - mae: 2.3404 - val_loss: 7.1975 - val_mae: 2.0178
Epoch 22/500
49/49 [==============================] - 1s 25ms/step - loss: 9.3913 - mae: 2.3864 - val_loss: 7.1437 - val_mae: 2.0081
Epoch 23/500
49/49 [==============================] - 1s 24ms/step - loss: 8.2620 - mae: 2.2456 - val_loss: 7.1363 - val_mae: 2.0066
Epoch 24/500
49/49 [==============================] - 1s 25ms/step - loss: 8.2001 - mae: 2.2339 - val_loss: 7.0943 - val_mae: 2.0011
Epoch 25/500
49/49 [==============================] - 2s 37ms/step - loss: 7.9151 - mae: 2.1691 - val_loss: 7.0702 - val_mae: 1.9983
Epoch 26/500
49/49 [==============================] - 2s 39ms/step - loss: 7.7760 - mae: 2.1497 - val_loss: 7.0308 - val_mae: 1.9901
Epoch 27/500
49/49 [==============================] - 2s 34ms/step - loss: 7.3079 - mae: 2.0799 - val_loss: 7.0016 - val_mae: 1.9855
Epoch 28/500
49/49 [==============================] - 1s 24ms/step - loss: 7.1954 - mae: 2.0644 - val_loss: 6.9758 - val_mae: 1.9840
Epoch 29/500
49/49 [==============================] - 1s 25ms/step - loss: 6.6855 - mae: 1.9930 - val_loss: 6.9370 - val_mae: 1.9796
Epoch 30/500
49/49 [==============================] - 1s 23ms/step - loss: 6.9411 - mae: 2.0134 - val_loss: 6.9193 - val_mae: 1.9790
Epoch 31/500
49/49 [==============================] - 1s 24ms/step - loss: 6.4107 - mae: 1.9316 - val_loss: 6.9096 - val_mae: 1.9789
Epoch 32/500
49/49 [==============================] - 1s 24ms/step - loss: 6.0207 - mae: 1.8624 - val_loss: 6.8841 - val_mae: 1.9763
Epoch 33/500
49/49 [==============================] - 1s 23ms/step - loss: 6.2004 - mae: 1.8757 - val_loss: 6.8587 - val_mae: 1.9736
Epoch 34/500
49/49 [==============================] - 1s 24ms/step - loss: 6.0841 - mae: 1.8324 - val_loss: 6.8428 - val_mae: 1.9724
Epoch 35/500
49/49 [==============================] - 1s 24ms/step - loss: 5.5891 - mae: 1.8083 - val_loss: 6.8359 - val_mae: 1.9738
Epoch 36/500
49/49 [==============================] - 2s 40ms/step - loss: 5.7786 - mae: 1.8058 - val_loss: 6.8138 - val_mae: 1.9716
Epoch 37/500
49/49 [==============================] - 2s 39ms/step - loss: 5.5074 - mae: 1.7505 - val_loss: 6.8063 - val_mae: 1.9709
Epoch 38/500
49/49 [==============================] - 1s 28ms/step - loss: 5.5479 - mae: 1.7662 - val_loss: 6.7650 - val_mae: 1.9648
Epoch 39/500
49/49 [==============================] - 1s 24ms/step - loss: 5.2039 - mae: 1.6980 - val_loss: 6.7664 - val_mae: 1.9660
Epoch 40/500
49/49 [==============================] - 1s 25ms/step - loss: 5.2874 - mae: 1.7196 - val_loss: 6.7460 - val_mae: 1.9636
Epoch 41/500
49/49 [==============================] - 1s 24ms/step - loss: 5.2867 - mae: 1.6788 - val_loss: 6.7307 - val_mae: 1.9607
Epoch 42/500
49/49 [==============================] - 1s 25ms/step - loss: 5.0894 - mae: 1.6615 - val_loss: 6.7063 - val_mae: 1.9561
Epoch 43/500
49/49 [==============================] - 1s 25ms/step - loss: 4.8726 - mae: 1.6069 - val_loss: 6.6873 - val_mae: 1.9525
Epoch 44/500
49/49 [==============================] - 1s 25ms/step - loss: 4.6708 - mae: 1.5747 - val_loss: 6.6710 - val_mae: 1.9493
Epoch 45/500
49/49 [==============================] - 1s 25ms/step - loss: 4.5467 - mae: 1.5504 - val_loss: 6.6578 - val_mae: 1.9468
Epoch 46/500
49/49 [==============================] - 2s 31ms/step - loss: 4.6180 - mae: 1.5636 - val_loss: 6.6498 - val_mae: 1.9435
Epoch 47/500
49/49 [==============================] - 2s 39ms/step - loss: 4.7395 - mae: 1.5710 - val_loss: 6.6398 - val_mae: 1.9418
Epoch 48/500
49/49 [==============================] - 2s 40ms/step - loss: 4.4284 - mae: 1.5363 - val_loss: 6.6200 - val_mae: 1.9384
Epoch 49/500
49/49 [==============================] - 1s 24ms/step - loss: 4.4592 - mae: 1.5374 - val_loss: 6.6108 - val_mae: 1.9370
Epoch 50/500
49/49 [==============================] - 1s 24ms/step - loss: 4.4687 - mae: 1.5147 - val_loss: 6.5924 - val_mae: 1.9338
Epoch 51/500
49/49 [==============================] - 1s 24ms/step - loss: 4.2277 - mae: 1.5003 - val_loss: 6.5790 - val_mae: 1.9325
Epoch 52/500
49/49 [==============================] - 1s 25ms/step - loss: 4.2775 - mae: 1.4749 - val_loss: 6.5587 - val_mae: 1.9299
Epoch 53/500
49/49 [==============================] - 1s 24ms/step - loss: 4.4446 - mae: 1.5086 - val_loss: 6.5327 - val_mae: 1.9248
Epoch 54/500
49/49 [==============================] - 1s 24ms/step - loss: 4.1709 - mae: 1.4626 - val_loss: 6.5241 - val_mae: 1.9228
Epoch 55/500
49/49 [==============================] - 1s 24ms/step - loss: 4.1437 - mae: 1.4446 - val_loss: 6.5140 - val_mae: 1.9204
Epoch 56/500
49/49 [==============================] - 1s 25ms/step - loss: 3.9987 - mae: 1.4454 - val_loss: 6.5187 - val_mae: 1.9210
Epoch 57/500
49/49 [==============================] - 2s 36ms/step - loss: 4.0825 - mae: 1.4497 - val_loss: 6.5193 - val_mae: 1.9214
Epoch 58/500
49/49 [==============================] - 2s 39ms/step - loss: 4.0586 - mae: 1.4110 - val_loss: 6.5179 - val_mae: 1.9207
Epoch 59/500
49/49 [==============================] - 2s 30ms/step - loss: 3.8754 - mae: 1.3968 - val_loss: 6.5053 - val_mae: 1.9180
Epoch 60/500
49/49 [==============================] - 1s 25ms/step - loss: 3.8891 - mae: 1.4031 - val_loss: 6.5010 - val_mae: 1.9178
Epoch 61/500
49/49 [==============================] - 1s 25ms/step - loss: 3.9725 - mae: 1.4108 - val_loss: 6.4860 - val_mae: 1.9139
Epoch 62/500
49/49 [==============================] - 1s 24ms/step - loss: 3.9284 - mae: 1.3929 - val_loss: 6.4842 - val_mae: 1.9145
Epoch 63/500
49/49 [==============================] - 1s 24ms/step - loss: 3.8529 - mae: 1.3806 - val_loss: 6.4782 - val_mae: 1.9140
Epoch 64/500
49/49 [==============================] - 1s 25ms/step - loss: 3.7103 - mae: 1.3444 - val_loss: 6.4736 - val_mae: 1.9138
Epoch 65/500
49/49 [==============================] - 1s 25ms/step - loss: 3.7225 - mae: 1.3483 - val_loss: 6.4732 - val_mae: 1.9129
Epoch 66/500
49/49 [==============================] - 1s 25ms/step - loss: 3.8266 - mae: 1.3601 - val_loss: 6.4693 - val_mae: 1.9130
Epoch 67/500
49/49 [==============================] - 2s 31ms/step - loss: 3.7672 - mae: 1.3499 - val_loss: 6.4713 - val_mae: 1.9131
Epoch 68/500
49/49 [==============================] - 2s 39ms/step - loss: 3.8136 - mae: 1.3577 - val_loss: 6.4648 - val_mae: 1.9107
Epoch 69/500
49/49 [==============================] - 2s 39ms/step - loss: 3.6085 - mae: 1.3318 - val_loss: 6.4620 - val_mae: 1.9097
Epoch 70/500
49/49 [==============================] - 1s 25ms/step - loss: 3.8274 - mae: 1.3652 - val_loss: 6.4695 - val_mae: 1.9116
Epoch 71/500
49/49 [==============================] - 1s 26ms/step - loss: 3.6697 - mae: 1.3348 - val_loss: 6.4630 - val_mae: 1.9100
Epoch 72/500
49/49 [==============================] - 1s 24ms/step - loss: 3.6726 - mae: 1.3294 - val_loss: 6.4572 - val_mae: 1.9086
Epoch 73/500
49/49 [==============================] - 1s 25ms/step - loss: 3.5677 - mae: 1.2978 - val_loss: 6.4672 - val_mae: 1.9103
Epoch 74/500
49/49 [==============================] - 1s 24ms/step - loss: 3.6030 - mae: 1.3081 - val_loss: 6.4547 - val_mae: 1.9089
Epoch 75/500
49/49 [==============================] - 1s 25ms/step - loss: 3.6422 - mae: 1.3148 - val_loss: 6.4607 - val_mae: 1.9110
Epoch 76/500
49/49 [==============================] - 1s 24ms/step - loss: 3.5756 - mae: 1.3037 - val_loss: 6.4596 - val_mae: 1.9107
Epoch 77/500
49/49 [==============================] - 1s 24ms/step - loss: 3.6398 - mae: 1.3099 - val_loss: 6.4587 - val_mae: 1.9108
Epoch 78/500
49/49 [==============================] - 2s 38ms/step - loss: 3.6485 - mae: 1.3151 - val_loss: 6.4549 - val_mae: 1.9101
Epoch 79/500
49/49 [==============================] - 2s 40ms/step - loss: 3.5126 - mae: 1.2881 - val_loss: 6.4517 - val_mae: 1.9095
Epoch 80/500
49/49 [==============================] - 2s 30ms/step - loss: 3.4713 - mae: 1.2769 - val_loss: 6.4457 - val_mae: 1.9084
Epoch 81/500
49/49 [==============================] - 1s 26ms/step - loss: 3.5149 - mae: 1.2923 - val_loss: 6.4387 - val_mae: 1.9067
Epoch 82/500
49/49 [==============================] - 1s 25ms/step - loss: 3.4944 - mae: 1.2786 - val_loss: 6.4452 - val_mae: 1.9099
Epoch 83/500
49/49 [==============================] - 1s 24ms/step - loss: 3.4444 - mae: 1.2596 - val_loss: 6.4385 - val_mae: 1.9090
Epoch 84/500
49/49 [==============================] - 1s 24ms/step - loss: 3.4386 - mae: 1.2715 - val_loss: 6.4433 - val_mae: 1.9109
Epoch 85/500
49/49 [==============================] - 1s 24ms/step - loss: 3.3768 - mae: 1.2501 - val_loss: 6.4409 - val_mae: 1.9106
Epoch 86/500
49/49 [==============================] - 1s 25ms/step - loss: 3.4484 - mae: 1.2704 - val_loss: 6.4279 - val_mae: 1.9079
Epoch 87/500
49/49 [==============================] - 1s 24ms/step - loss: 3.5827 - mae: 1.2982 - val_loss: 6.4133 - val_mae: 1.9056
Epoch 88/500
49/49 [==============================] - 1s 30ms/step - loss: 3.4165 - mae: 1.2530 - val_loss: 6.4201 - val_mae: 1.9076
Epoch 89/500
49/49 [==============================] - 2s 39ms/step - loss: 3.3561 - mae: 1.2377 - val_loss: 6.4294 - val_mae: 1.9087
Epoch 90/500
49/49 [==============================] - 2s 38ms/step - loss: 3.4136 - mae: 1.2434 - val_loss: 6.4217 - val_mae: 1.9060
Epoch 91/500
49/49 [==============================] - 1s 25ms/step - loss: 3.3603 - mae: 1.2565 - val_loss: 6.4312 - val_mae: 1.9090
Epoch 92/500
49/49 [==============================] - 1s 26ms/step - loss: 3.4217 - mae: 1.2676 - val_loss: 6.4208 - val_mae: 1.9063
Epoch 93/500
49/49 [==============================] - 1s 25ms/step - loss: 3.3177 - mae: 1.2292 - val_loss: 6.4154 - val_mae: 1.9052
Epoch 94/500
49/49 [==============================] - 1s 25ms/step - loss: 3.3508 - mae: 1.2456 - val_loss: 6.4169 - val_mae: 1.9061
Epoch 95/500
49/49 [==============================] - 1s 24ms/step - loss: 3.3620 - mae: 1.2332 - val_loss: 6.4113 - val_mae: 1.9057
Epoch 96/500
49/49 [==============================] - 1s 24ms/step - loss: 3.3059 - mae: 1.2307 - val_loss: 6.4099 - val_mae: 1.9052
Epoch 97/500
49/49 [==============================] - 1s 24ms/step - loss: 3.2456 - mae: 1.2239 - val_loss: 6.4165 - val_mae: 1.9067
Epoch 98/500
49/49 [==============================] - 1s 25ms/step - loss: 3.2913 - mae: 1.2339 - val_loss: 6.4129 - val_mae: 1.9060
Epoch 99/500
49/49 [==============================] - 2s 38ms/step - loss: 3.2746 - mae: 1.2304 - val_loss: 6.4062 - val_mae: 1.9053
Epoch 100/500
49/49 [==============================] - 2s 40ms/step - loss: 3.2073 - mae: 1.2065 - val_loss: 6.4135 - val_mae: 1.9061
Epoch 101/500
49/49 [==============================] - 2s 32ms/step - loss: 3.2865 - mae: 1.2492 - val_loss: 6.4112 - val_mae: 1.9056
Epoch 102/500
49/49 [==============================] - 1s 25ms/step - loss: 3.1876 - mae: 1.2183 - val_loss: 6.4091 - val_mae: 1.9051
Epoch 103/500
49/49 [==============================] - 1s 24ms/step - loss: 3.2006 - mae: 1.2088 - val_loss: 6.4106 - val_mae: 1.9048
Epoch 104/500
49/49 [==============================] - 1s 24ms/step - loss: 3.2369 - mae: 1.2128 - val_loss: 6.4072 - val_mae: 1.9039
Epoch 105/500
49/49 [==============================] - 1s 24ms/step - loss: 3.1768 - mae: 1.2116 - val_loss: 6.3962 - val_mae: 1.9031
Epoch 106/500
49/49 [==============================] - 1s 24ms/step - loss: 3.2115 - mae: 1.2204 - val_loss: 6.4058 - val_mae: 1.9045
Epoch 107/500
49/49 [==============================] - 1s 24ms/step - loss: 3.2317 - mae: 1.2178 - val_loss: 6.4031 - val_mae: 1.9045
Epoch 108/500
49/49 [==============================] - 1s 25ms/step - loss: 3.1023 - mae: 1.1918 - val_loss: 6.3976 - val_mae: 1.9025
Epoch 109/500
49/49 [==============================] - 1s 28ms/step - loss: 3.2555 - mae: 1.2248 - val_loss: 6.3733 - val_mae: 1.8965
Epoch 110/500
49/49 [==============================] - 2s 38ms/step - loss: 3.0978 - mae: 1.2073 - val_loss: 6.3876 - val_mae: 1.8999
Epoch 111/500
49/49 [==============================] - 2s 40ms/step - loss: 3.1049 - mae: 1.2026 - val_loss: 6.3815 - val_mae: 1.9002
Epoch 112/500
49/49 [==============================] - 1s 28ms/step - loss: 2.9761 - mae: 1.1658 - val_loss: 6.3823 - val_mae: 1.9003
Epoch 113/500
49/49 [==============================] - 1s 24ms/step - loss: 3.1203 - mae: 1.1989 - val_loss: 6.4012 - val_mae: 1.9049
Epoch 114/500
49/49 [==============================] - 1s 25ms/step - loss: 3.0839 - mae: 1.1979 - val_loss: 6.3821 - val_mae: 1.9003
Epoch 115/500
49/49 [==============================] - 1s 24ms/step - loss: 3.0819 - mae: 1.1949 - val_loss: 6.3816 - val_mae: 1.8993
Epoch 116/500
49/49 [==============================] - 1s 24ms/step - loss: 3.0271 - mae: 1.1793 - val_loss: 6.3909 - val_mae: 1.9013
Epoch 117/500
49/49 [==============================] - 1s 25ms/step - loss: 3.0657 - mae: 1.1960 - val_loss: 6.3911 - val_mae: 1.9015
Epoch 118/500
49/49 [==============================] - 1s 25ms/step - loss: 3.0145 - mae: 1.1873 - val_loss: 6.3855 - val_mae: 1.8995
Epoch 119/500
48/49 [============================>.] - ETA: 0s - loss: 3.0004 - mae: 1.1843Restoring model weights from the end of the best epoch: 109.
49/49 [==============================] - 1s 26ms/step - loss: 2.9885 - mae: 1.1833 - val_loss: 6.3905 - val_mae: 1.9015
Epoch 119: early stopping
13/13 [==============================] - 0s 12ms/step - loss: 6.3733 - mae: 1.8965
Test Loss: 6.37326717376709
Test MAE: 1.8964616060256958
13/13 [==============================] - 0s 12ms/step

Predicting Fractional differences¶

Using GAF representation of the fractionally-differenced time series, let us do the forecasting.

target_variable = 'Fractional'
X_data_train, X_data_test, y_data_train, y_data_test = train_test_data_cnn(target_variable)
cnn(target_variable, X_data_train, X_data_test, y_data_train, y_data_test)

Model: "sequential_6"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d_6 (Conv2D)           (None, 28, 28, 16)        160       
                                                                 
 max_pooling2d_6 (MaxPoolin  (None, 14, 14, 16)        0         
 g2D)                                                            
                                                                 
 conv2d_7 (Conv2D)           (None, 12, 12, 32)        4640      
                                                                 
 max_pooling2d_7 (MaxPoolin  (None, 6, 6, 32)          0         
 g2D)                                                            
                                                                 
 conv2d_8 (Conv2D)           (None, 4, 4, 64)          18496     
                                                                 
 max_pooling2d_8 (MaxPoolin  (None, 2, 2, 64)          0         
 g2D)                                                            
                                                                 
 flatten_2 (Flatten)         (None, 256)               0         
                                                                 
 dense_16 (Dense)            (None, 1024)              263168    
                                                                 
 dropout_10 (Dropout)        (None, 1024)              0         
                                                                 
 dense_17 (Dense)            (None, 1)                 1025      
                                                                 
=================================================================
Total params: 287489 (1.10 MB)
Trainable params: 287489 (1.10 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
Epoch 1/500
49/49 [==============================] - 4s 58ms/step - loss: 27.2632 - mae: 4.1061 - val_loss: 8.7718 - val_mae: 2.2305
Epoch 2/500
49/49 [==============================] - 2s 47ms/step - loss: 22.9806 - mae: 3.7648 - val_loss: 7.9533 - val_mae: 2.0978
Epoch 3/500
49/49 [==============================] - 2s 43ms/step - loss: 19.1490 - mae: 3.4224 - val_loss: 7.8512 - val_mae: 2.0873
Epoch 4/500
49/49 [==============================] - 4s 78ms/step - loss: 17.1802 - mae: 3.2508 - val_loss: 8.2565 - val_mae: 2.1581
Epoch 5/500
49/49 [==============================] - 3s 66ms/step - loss: 16.4497 - mae: 3.2072 - val_loss: 7.9958 - val_mae: 2.1179
Epoch 6/500
49/49 [==============================] - 2s 47ms/step - loss: 13.8075 - mae: 2.9340 - val_loss: 7.8268 - val_mae: 2.0903
Epoch 7/500
49/49 [==============================] - 2s 44ms/step - loss: 13.1320 - mae: 2.8384 - val_loss: 8.0408 - val_mae: 2.1309
Epoch 8/500
49/49 [==============================] - 1s 25ms/step - loss: 11.7941 - mae: 2.6805 - val_loss: 7.8133 - val_mae: 2.0932
Epoch 9/500
49/49 [==============================] - 1s 25ms/step - loss: 10.8261 - mae: 2.5701 - val_loss: 7.6529 - val_mae: 2.0685
Epoch 10/500
49/49 [==============================] - 1s 24ms/step - loss: 10.2811 - mae: 2.4927 - val_loss: 7.5584 - val_mae: 2.0558
Epoch 11/500
49/49 [==============================] - 1s 28ms/step - loss: 9.0309 - mae: 2.3549 - val_loss: 7.8213 - val_mae: 2.1040
Epoch 12/500
49/49 [==============================] - 2s 39ms/step - loss: 8.8754 - mae: 2.3131 - val_loss: 7.6764 - val_mae: 2.0812
Epoch 13/500
49/49 [==============================] - 2s 40ms/step - loss: 8.5763 - mae: 2.2777 - val_loss: 7.4692 - val_mae: 2.0473
Epoch 14/500
49/49 [==============================] - 1s 25ms/step - loss: 8.0882 - mae: 2.2155 - val_loss: 7.5572 - val_mae: 2.0638
Epoch 15/500
49/49 [==============================] - 1s 25ms/step - loss: 8.0623 - mae: 2.1760 - val_loss: 7.4928 - val_mae: 2.0532
Epoch 16/500
49/49 [==============================] - 1s 25ms/step - loss: 7.5237 - mae: 2.1362 - val_loss: 7.6080 - val_mae: 2.0771
Epoch 17/500
49/49 [==============================] - 1s 24ms/step - loss: 7.0219 - mae: 2.0536 - val_loss: 7.5543 - val_mae: 2.0695
Epoch 18/500
49/49 [==============================] - 1s 25ms/step - loss: 6.9243 - mae: 1.9897 - val_loss: 7.3718 - val_mae: 2.0365
Epoch 19/500
49/49 [==============================] - 1s 23ms/step - loss: 6.5957 - mae: 1.9561 - val_loss: 7.5715 - val_mae: 2.0735
Epoch 20/500
49/49 [==============================] - 1s 25ms/step - loss: 6.2463 - mae: 1.8917 - val_loss: 7.5063 - val_mae: 2.0628
Epoch 21/500
49/49 [==============================] - 1s 25ms/step - loss: 5.8436 - mae: 1.8475 - val_loss: 7.5235 - val_mae: 2.0676
Epoch 22/500
49/49 [==============================] - 2s 34ms/step - loss: 6.3622 - mae: 1.9060 - val_loss: 7.3802 - val_mae: 2.0423
Epoch 23/500
49/49 [==============================] - 2s 38ms/step - loss: 5.6636 - mae: 1.7969 - val_loss: 7.4763 - val_mae: 2.0605
Epoch 24/500
49/49 [==============================] - 2s 34ms/step - loss: 5.5760 - mae: 1.7750 - val_loss: 7.4002 - val_mae: 2.0483
Epoch 25/500
49/49 [==============================] - 1s 25ms/step - loss: 5.5196 - mae: 1.7775 - val_loss: 7.4431 - val_mae: 2.0579
Epoch 26/500
49/49 [==============================] - 1s 23ms/step - loss: 5.7386 - mae: 1.7769 - val_loss: 7.3950 - val_mae: 2.0502
Epoch 27/500
49/49 [==============================] - 1s 24ms/step - loss: 5.4108 - mae: 1.7526 - val_loss: 7.3286 - val_mae: 2.0391
Epoch 28/500
49/49 [==============================] - 1s 24ms/step - loss: 5.3978 - mae: 1.7421 - val_loss: 7.4831 - val_mae: 2.0687
Epoch 29/500
49/49 [==============================] - 1s 24ms/step - loss: 4.9128 - mae: 1.6634 - val_loss: 7.3984 - val_mae: 2.0529
Epoch 30/500
49/49 [==============================] - 1s 25ms/step - loss: 5.3629 - mae: 1.6999 - val_loss: 7.3174 - val_mae: 2.0382
Epoch 31/500
49/49 [==============================] - 1s 25ms/step - loss: 4.9905 - mae: 1.6608 - val_loss: 7.3246 - val_mae: 2.0428
Epoch 32/500
49/49 [==============================] - 1s 25ms/step - loss: 4.7972 - mae: 1.6075 - val_loss: 7.2593 - val_mae: 2.0312
Epoch 33/500
49/49 [==============================] - 2s 39ms/step - loss: 4.8077 - mae: 1.6111 - val_loss: 7.3146 - val_mae: 2.0432
Epoch 34/500
49/49 [==============================] - 2s 41ms/step - loss: 4.7719 - mae: 1.6113 - val_loss: 7.2402 - val_mae: 2.0272
Epoch 35/500
49/49 [==============================] - 1s 26ms/step - loss: 4.8240 - mae: 1.6026 - val_loss: 7.2577 - val_mae: 2.0329
Epoch 36/500
49/49 [==============================] - 1s 25ms/step - loss: 4.4942 - mae: 1.5663 - val_loss: 7.3034 - val_mae: 2.0427
Epoch 37/500
49/49 [==============================] - 1s 25ms/step - loss: 4.3938 - mae: 1.5325 - val_loss: 7.2754 - val_mae: 2.0377
Epoch 38/500
49/49 [==============================] - 1s 25ms/step - loss: 4.4004 - mae: 1.5367 - val_loss: 7.3166 - val_mae: 2.0472
Epoch 39/500
49/49 [==============================] - 1s 24ms/step - loss: 4.2610 - mae: 1.5243 - val_loss: 7.2270 - val_mae: 2.0289
Epoch 40/500
49/49 [==============================] - 1s 24ms/step - loss: 4.2804 - mae: 1.5107 - val_loss: 7.2212 - val_mae: 2.0293
Epoch 41/500
49/49 [==============================] - 1s 25ms/step - loss: 4.4193 - mae: 1.5499 - val_loss: 7.1857 - val_mae: 2.0235
Epoch 42/500
49/49 [==============================] - 1s 23ms/step - loss: 4.2008 - mae: 1.4945 - val_loss: 7.2057 - val_mae: 2.0277
Epoch 43/500
49/49 [==============================] - 2s 32ms/step - loss: 4.2781 - mae: 1.5155 - val_loss: 7.2533 - val_mae: 2.0361
Epoch 44/500
49/49 [==============================] - 2s 39ms/step - loss: 4.2845 - mae: 1.4914 - val_loss: 7.2414 - val_mae: 2.0350
Epoch 45/500
49/49 [==============================] - 2s 35ms/step - loss: 4.2126 - mae: 1.4826 - val_loss: 7.2462 - val_mae: 2.0358
Epoch 46/500
49/49 [==============================] - 1s 25ms/step - loss: 4.0782 - mae: 1.4492 - val_loss: 7.1880 - val_mae: 2.0239
Epoch 47/500
49/49 [==============================] - 1s 26ms/step - loss: 3.8964 - mae: 1.4305 - val_loss: 7.2058 - val_mae: 2.0278
Epoch 48/500
49/49 [==============================] - 1s 24ms/step - loss: 4.1027 - mae: 1.4508 - val_loss: 7.2170 - val_mae: 2.0300
Epoch 49/500
49/49 [==============================] - 1s 25ms/step - loss: 4.0590 - mae: 1.4465 - val_loss: 7.1153 - val_mae: 2.0109
Epoch 50/500
49/49 [==============================] - 1s 25ms/step - loss: 3.8881 - mae: 1.4271 - val_loss: 7.1527 - val_mae: 2.0190
Epoch 51/500
49/49 [==============================] - 1s 25ms/step - loss: 3.8569 - mae: 1.4202 - val_loss: 7.1854 - val_mae: 2.0260
Epoch 52/500
49/49 [==============================] - 1s 24ms/step - loss: 3.8923 - mae: 1.4013 - val_loss: 7.2146 - val_mae: 2.0317
Epoch 53/500
49/49 [==============================] - 1s 28ms/step - loss: 3.8531 - mae: 1.3976 - val_loss: 7.1449 - val_mae: 2.0195
Epoch 54/500
49/49 [==============================] - 2s 38ms/step - loss: 3.8972 - mae: 1.4051 - val_loss: 7.0831 - val_mae: 2.0087
Epoch 55/500
49/49 [==============================] - 2s 40ms/step - loss: 3.8561 - mae: 1.3865 - val_loss: 7.1271 - val_mae: 2.0185
Epoch 56/500
49/49 [==============================] - 1s 27ms/step - loss: 3.6818 - mae: 1.3785 - val_loss: 7.0940 - val_mae: 2.0110
Epoch 57/500
49/49 [==============================] - 1s 23ms/step - loss: 3.7957 - mae: 1.3712 - val_loss: 7.1268 - val_mae: 2.0170
Epoch 58/500
49/49 [==============================] - 1s 25ms/step - loss: 3.7780 - mae: 1.3846 - val_loss: 7.1206 - val_mae: 2.0145
Epoch 59/500
49/49 [==============================] - 1s 24ms/step - loss: 3.6963 - mae: 1.3770 - val_loss: 7.1017 - val_mae: 2.0113
Epoch 60/500
49/49 [==============================] - 1s 23ms/step - loss: 3.6964 - mae: 1.3641 - val_loss: 7.1122 - val_mae: 2.0134
Epoch 61/500
49/49 [==============================] - 1s 25ms/step - loss: 3.8366 - mae: 1.3982 - val_loss: 7.0917 - val_mae: 2.0104
Epoch 62/500
49/49 [==============================] - 1s 24ms/step - loss: 3.7369 - mae: 1.3713 - val_loss: 7.1330 - val_mae: 2.0176
Epoch 63/500
49/49 [==============================] - 1s 25ms/step - loss: 3.7124 - mae: 1.3605 - val_loss: 7.0743 - val_mae: 2.0081
Epoch 64/500
49/49 [==============================] - 2s 31ms/step - loss: 3.5595 - mae: 1.3457 - val_loss: 7.0981 - val_mae: 2.0124
Epoch 65/500
49/49 [==============================] - 2s 38ms/step - loss: 3.5892 - mae: 1.3393 - val_loss: 7.0729 - val_mae: 2.0077
Epoch 66/500
49/49 [==============================] - 2s 33ms/step - loss: 3.5020 - mae: 1.3272 - val_loss: 7.0837 - val_mae: 2.0106
Epoch 67/500
49/49 [==============================] - 1s 25ms/step - loss: 3.4844 - mae: 1.3182 - val_loss: 7.0488 - val_mae: 2.0041
Epoch 68/500
49/49 [==============================] - 1s 24ms/step - loss: 3.4587 - mae: 1.3091 - val_loss: 7.0958 - val_mae: 2.0121
Epoch 69/500
49/49 [==============================] - 1s 25ms/step - loss: 3.4738 - mae: 1.3216 - val_loss: 7.0826 - val_mae: 2.0091
Epoch 70/500
49/49 [==============================] - 1s 27ms/step - loss: 3.5156 - mae: 1.3124 - val_loss: 7.0831 - val_mae: 2.0097
Epoch 71/500
49/49 [==============================] - 1s 25ms/step - loss: 3.4228 - mae: 1.3132 - val_loss: 7.0800 - val_mae: 2.0089
Epoch 72/500
49/49 [==============================] - 1s 25ms/step - loss: 3.3759 - mae: 1.3035 - val_loss: 7.0478 - val_mae: 2.0033
Epoch 73/500
49/49 [==============================] - 1s 24ms/step - loss: 3.4668 - mae: 1.3268 - val_loss: 7.1114 - val_mae: 2.0144
Epoch 74/500
49/49 [==============================] - 1s 27ms/step - loss: 3.4224 - mae: 1.2965 - val_loss: 7.0960 - val_mae: 2.0098
Epoch 75/500
49/49 [==============================] - 2s 40ms/step - loss: 3.3337 - mae: 1.2860 - val_loss: 7.0511 - val_mae: 2.0023
Epoch 76/500
49/49 [==============================] - 2s 35ms/step - loss: 3.3277 - mae: 1.2943 - val_loss: 7.0756 - val_mae: 2.0069
Epoch 77/500
49/49 [==============================] - 1s 25ms/step - loss: 3.3281 - mae: 1.3023 - val_loss: 7.0694 - val_mae: 2.0065
Epoch 78/500
49/49 [==============================] - 1s 25ms/step - loss: 3.3761 - mae: 1.3030 - val_loss: 7.1051 - val_mae: 2.0130
Epoch 79/500
49/49 [==============================] - 1s 23ms/step - loss: 3.3250 - mae: 1.2919 - val_loss: 7.0486 - val_mae: 2.0022
Epoch 80/500
49/49 [==============================] - 1s 24ms/step - loss: 3.3683 - mae: 1.3022 - val_loss: 7.0767 - val_mae: 2.0068
Epoch 81/500
49/49 [==============================] - 1s 25ms/step - loss: 3.2008 - mae: 1.2610 - val_loss: 7.1086 - val_mae: 2.0120
Epoch 82/500
49/49 [==============================] - 1s 24ms/step - loss: 3.2799 - mae: 1.2815 - val_loss: 7.0474 - val_mae: 2.0008
Epoch 83/500
49/49 [==============================] - 1s 24ms/step - loss: 3.2799 - mae: 1.2784 - val_loss: 7.0612 - val_mae: 2.0031
Epoch 84/500
49/49 [==============================] - 1s 24ms/step - loss: 3.2323 - mae: 1.2726 - val_loss: 7.0768 - val_mae: 2.0070
Epoch 85/500
49/49 [==============================] - 2s 38ms/step - loss: 3.1961 - mae: 1.2663 - val_loss: 7.1280 - val_mae: 2.0160
Epoch 86/500
49/49 [==============================] - 2s 39ms/step - loss: 3.3683 - mae: 1.2960 - val_loss: 7.0893 - val_mae: 2.0087
Epoch 87/500
49/49 [==============================] - 2s 33ms/step - loss: 3.1709 - mae: 1.2627 - val_loss: 7.0376 - val_mae: 2.0006
Epoch 88/500
49/49 [==============================] - 1s 25ms/step - loss: 3.1528 - mae: 1.2553 - val_loss: 7.1155 - val_mae: 2.0143
Epoch 89/500
49/49 [==============================] - 1s 25ms/step - loss: 3.2200 - mae: 1.2541 - val_loss: 7.0671 - val_mae: 2.0066
Epoch 90/500
49/49 [==============================] - 1s 24ms/step - loss: 3.1239 - mae: 1.2533 - val_loss: 7.0772 - val_mae: 2.0091
Epoch 91/500
49/49 [==============================] - 1s 25ms/step - loss: 3.2307 - mae: 1.2607 - val_loss: 7.1208 - val_mae: 2.0155
Epoch 92/500
49/49 [==============================] - 1s 25ms/step - loss: 3.1392 - mae: 1.2522 - val_loss: 7.0389 - val_mae: 1.9995
Epoch 93/500
49/49 [==============================] - 1s 25ms/step - loss: 3.0783 - mae: 1.2528 - val_loss: 7.0977 - val_mae: 2.0109
Epoch 94/500
49/49 [==============================] - 1s 24ms/step - loss: 3.1314 - mae: 1.2427 - val_loss: 7.1401 - val_mae: 2.0180
Epoch 95/500
49/49 [==============================] - 1s 28ms/step - loss: 3.0079 - mae: 1.2240 - val_loss: 7.0610 - val_mae: 2.0040
Epoch 96/500
49/49 [==============================] - 2s 38ms/step - loss: 2.9993 - mae: 1.2074 - val_loss: 7.1008 - val_mae: 2.0124
Epoch 97/500
47/49 [===========================>..] - ETA: 0s - loss: 2.9634 - mae: 1.2323Restoring model weights from the end of the best epoch: 87.
49/49 [==============================] - 2s 39ms/step - loss: 2.9333 - mae: 1.2250 - val_loss: 7.0774 - val_mae: 2.0081
Epoch 97: early stopping
13/13 [==============================] - 0s 11ms/step - loss: 7.0376 - mae: 2.0006
Test Loss: 7.037605285644531
Test MAE: 2.0006046295166016
13/13 [==============================] - 0s 8ms/step

Our evaluation of the models is based on the Mean absolute error on the test set, the loss is the mean squared error. As summarized below

MLP from time series
- Test Loss: 0.1447
- Test MAE: 0.31504
MLP from stationary time series (using differentiation)
- Test Loss: 0.02199
- Test MAE: 0.1176
MLP from stationary time series (using partial differentiation)
- Test Loss: 0.0185
- Test MAE: 0.10489

In this representation the Convolution neural networks do better, the best performance was obtained with the data that was partially differentiated, The model does not seem to do too good on the test set, this might be due to the newtwork not being complex enough or the architecture of the CNN might be too simple for the comple structure of the time series.

While the metrics used indicate that using a MLP on time series might be a better approach than translating the time series into an image to train a CNN.

#AIandDSSkills

0 comments

12 views

AI and Data Science

Master the art of AI and Data Science.

Global AI and Data Science

Time-series data analysis using Deep Learning

By Randhir Singh posted Wed May 22, 2024 07:34 PM

Time-series data analysis using Deep Learning¶

Differencing¶

Fractional Differencing¶

Multi-Layer Perceptron model (MLP or Artificial Neural Network)¶

Predicting Price Levels¶

Predicting Returns¶

Predicting Fractional differences¶

Convolutional Neural Network (CNN)¶

Predicting Price Levels¶

Predicting Returns¶

Predicting Fractional differences¶

Permalink