# An Agent AI that Trades Stocks

# Intro

I was looking for ideas or environments to train an agent, when I found this stocks trading env

With this training environment, you can teach your AI to trade stocks, as a RL task.

It is easy as importing a module:

1 | from gym_anytrading.envs import TradingEnv, ForexEnv, StocksEnv, Actions, Positions |

1 | print("env information:") |

env information:

shape: (10, 2)

df.shape: (2335, 6)

prices.shape: (60,)

signal_features.shape: (60, 2)

max_possible_profit: 1.351611419906775

# The Approach

I went for a simple linear model, but with a *twist*. I, again, used Gates. As for the context, I used the total profit (the multiplier from the base ammount of money) squared, to accentuate extremes. I think this is how it figured out the trick it did later on.

I took the prediction of multiple old timesteps, and fed it to the model, like a Many-to-One architecture.

Finally, for the DQN, I used a deque for the memory, and stored the cuda Tensors for less RAM usage.

# The Framework

Here, I used pyTorch, which I find better for prototyping, and now, I’ve switched to it.

# The Model

Here’s a simple model I used.

1 | self.grunit = nn.GRU(observation.shape[1], r_hidden) |

1 | dqn = DQN(64, 32) |

In total, this is 15,000 params

And the Hyperparameters I chose for the DQN

1 | EPISODES = 300 |

For the activation function, I used `gelu`

for the temporal layer, `sigmoid`

for the gate, and `softmax`

for the probabilistic output.

# Results

Here, I found something interesting. The Agent **BROKE** the environment. Take a look at the profit over time, in multiplication factor (1 being no profit):

Well, actually, it found a method to simply earn consistantly 15% of what it invested.

Strangely enough, its average reward was 0.000.