I was looking for ideas or environments to train an agent, when I found this stocks trading env
With this training environment, you can teach your AI to trade stocks, as a RL task.
It is easy as importing a module:
from gym_anytrading.envs import TradingEnv, ForexEnv, StocksEnv, Actions, Positions
shape: (10, 2)
df.shape: (2335, 6)
signal_features.shape: (60, 2)
I went for a simple linear model, but with a twist. I, again, used Gates. As for the context, I used the total profit (the multiplier from the base ammount of money) squared, to accentuate extremes. I think this is how it figured out the trick it did later on.
I took the prediction of multiple old timesteps, and fed it to the model, like a Many-to-One architecture.
Finally, for the DQN, I used a deque for the memory, and stored the cuda Tensors for less RAM usage.
Here, I used pyTorch, which I find better for prototyping, and now, I’ve switched to it.
Here’s a simple model I used.
self.grunit = nn.GRU(observation.shape, r_hidden)
dqn = DQN(64, 32)
In total, this is 15,000 params
And the Hyperparameters I chose for the DQN
EPISODES = 300
For the activation function, I used
gelu for the temporal layer,
sigmoid for the gate, and
softmax for the probabilistic output.
Here, I found something interesting. The Agent BROKE the environment. Take a look at the profit over time, in multiplication factor (1 being no profit):
Well, actually, it found a method to simply earn consistantly 15% of what it invested.
Strangely enough, its average reward was 0.000.