Mean directional accuracy of time series forecast

Mean directional accuracy of time series forecast

by

Time series prediction performance measures provide a summary of the skill and capability of the forecast model that made the predictions. There are many different performance measures to choose from. In this post, we will study MDA or mean directional accuracy of time series forecasts.

Mean directional accuracy (MDA)

MDA is a measure of the prediction accuracy of a forecasting method in statistics. It compares the forecast direction (upward or downward) to the actual realized direction. It is a popular metric for forecasting performance in economics and finance. MDA is used where we are often interested only in the directional movement of variables of interest.

Examples:

As an example, a trader wants to know if the price of a share has an increasing direction or decreasing trend.

Another example can be found in financial planning where the user wants to know if the demand has an increasing direction or decreasing trend.

Another example, a hotel is likely more interested in predicting whether the degree of cancellations will increase/decrease in a particular week — as opposed to the precise number of cancellations. In this regard, the mean directional accuracy is used to determine the degree to which the model accurately forecasts the directional changes in cancellation frequency from week to week.

The formula for calculating the MDA- mean directional accuracy of time series forecast

MDA measures how often the predicted direction of a time series matches the actual direction of the time series. To calculate MDA, you look at the signs of the differences between consecutive actual values and the signs of the differences between consecutive predicted values. If the signs are the same (i.e., both positive or both negative), that means the predicted direction matches the actual direction. You count how many times this happens, and divide by the total number of possible comparisons (which is one less than the length of the time series, because you can’t compare the first value to anything). This gives you the MDA value, which ranges from 0 to 1, with 1 indicating perfect directional accuracy.

MDA = Number of times the signs of the differences between consecutive actual values are the same as the signs of the differences between consecutive predicted values / (N – 1)

where N is the length of the time series.

In mathematical notation, this can be expressed as:

MDA = sum(i=2 to N) sign(actual[i] – actual[i-1]) * sign(predicted[i] – predicted[i-1]) / (N – 1)

where sign(x) returns the sign of x (i.e., -1 if x < 0, 0 if x == 0, and 1 if x > 0).

Python code to calculate the MDA- mean directional accuracy of time series forecast

import numpy as np

def mda(actual, predicted):
    """
    Calculates the Mean Directional Accuracy (MDA) for two time series.
    
    Parameters:
    actual (array-like): The actual values for the time series.
    predicted (array-like): The predicted values for the time series.
    
    Returns:
    float: The MDA value.
    """
    actual = np.array(actual)
    predicted = np.array(predicted)
    
    # calculate the signs of the differences between consecutive values
    actual_diff = np.diff(actual)
    actual_signs = np.sign(actual_diff)
    predicted_diff = np.diff(predicted)
    predicted_signs = np.sign(predicted_diff)
    
    # count the number of times the signs are the same
    num_correct = np.sum(actual_signs == predicted_signs)
    
    # calculate the MDA value
    mda = num_correct / (len(actual) - 1)
    
    return mda

You can call this function with two arrays of time series values (one for the actual values, and one for the predicted values), and it will return the MDA value. The MDA ranges from 0 to 1, where a value of 1 indicates perfect directional accuracy. If MDA for 5 observations is 0.6 i.e. 3/5. It means 3 out of 5 directions were predicted correctly.

MDA tests the ability of the underlying model to predict the direction of change than the magnitude of the forecasting error.

Popular time series performance measures

  • Forecast Error (or Residual Forecast Error): forecast_error = expected_value – predicted_value
  • Mean Forecast Error (or Forecast Bias): mean_forecast_error = mean(forecast_error)
  • Mean Absolute Error: mean_absolute_error = mean( abs(forecast_error) )
  • Mean Squared Error: mean_squared_error = mean(forecast_error^2)
  • Root Mean Squared Error: rmse = sqrt(mean_squared_error)

Conclusion

The article “Mean Directional Accuracy of Time Series Forecast” discusses a common evaluation metric used in time series forecasting called the Mean Directional Accuracy (MDA). The MDA measures the accuracy of the direction of change in the forecasted values, rather than just the magnitude of the errors.

I highly recommend checking out this incredibly informative and engaging professional certificate Training by Google on Coursera:

Google Advanced Data Analytics Professional Certificate

There are 7 Courses in this Professional Certificate that can also be taken separately.

  1. Foundations of Data Science: Approx. 21 hours to complete. SKILLS YOU WILL GAIN: Sharing Insights With Stakeholders, Effective Written Communication, Asking Effective Questions, Cross-Functional Team Dynamics, and Project Management.
  2. Get Started with Python: Approx. 25 hours to complete. SKILLS YOU WILL GAIN: Using Comments to Enhance Code Readability, Python Programming, Jupyter Notebook, Data Visualization (DataViz), and Coding.
  3. Go Beyond the Numbers: Translate Data into Insights: Approx. 28 hours to complete. SKILLS YOU WILL GAIN: Python Programming, Tableau Software, Data Visualization (DataViz), Effective Communication, and Exploratory Data Analysis.
  4. The Power of Statistics: Approx. 33 hours to complete. SKILLS YOU WILL GAIN: Statistical Analysis, Python Programming, Effective Communication, Statistical Hypothesis Testing, and Probability Distribution.
  5. Regression Analysis: Simplify Complex Data Relationships: Approx. 28 hours to complete. SKILLS YOU WILL GAIN: Predictive Modelling, Statistical Analysis, Python Programming, Effective Communication, and regression modeling.
  6. The Nuts and Bolts of Machine Learning: Approx. 33 hours to complete. SKILLS YOU WILL GAIN: Predictive Modelling, Machine Learning, Python Programming, Stack Overflow, and Effective Communication.
  7. Google Advanced Data Analytics Capstone: Approx. 9 hours to complete. SKILLS YOU WILL GAIN: Executive Summaries, Machine Learning, Python Programming, Technical Interview Preparation, and Data Analysis.

It could be the perfect way to take your skills to the next level! When it comes to investing, there’s no better investment than investing in yourself and your education. Don’t hesitate – go ahead and take the leap. The benefits of learning and self-improvement are immeasurable.

You may also like:

Check out the table of contents for Product Management and Data Science to explore these topics further.

Curious about how product managers can utilize Bhagwad Gita’s principles to tackle difficulties? Give this super short book a shot. This will certainly support my work.

Leave a Comment