DEVELOPMENT OF AN AUTONOMOU AGENT FOR A NUMBER STRATEGY GAME USING DEEP Q-NETWORK

Abstract

Deep Q-Networks (DQNs) have emerged as a pivotal reinforcement learning algorithm for training autonomous agents in complex decision-making tasks. This study investigates the application of Deep Q-Networks in Numero, a number strategy game that requires logical reasoning and iterative feedback processing. Numero is a number strategy game where players predict an opponent's secret four-digit number in the fewest steps possible by analysing feedback and refining strategies. The study explores Numero's unique challenges, such as sparse reward structures, high-dimensional state-action spaces, and non-deterministic feedback mechanisms. To address these challenges, a Deep Q-Network algorithm augmented with Prioritised Experience Replay(PER) was designed and developed to enhance sample efficiency by prioritising critical experiences during training. The autonomous agent interacts with the custom environment, sampling mini-batches from the replay buffer, performing backpropagation, and updating Q-values to improve decision-making. Hyperparameters, such as learning rate, discount factor, replay buffer and exploration rate, were tuned to optimise the agent's learning efficiency. Comparative analysis was conducted using Reservoir Sampling without Replacement and the Minimax algorithm as a baseline approach. Experimental results show that the algorithm achieved a higher success rate (correctly predicted numbers) and faster convergence than Minimax, reducing the average number of steps required to guess the secret number by more than 100%. Additionally, this algorithm demonstrated superior adaptability in handling dynamic feedback, outperforming Reservoir sampling in long-term decision-making. These findings reveal the effectiveness of Deep Q-Networks in structured feedback-driven environments, suggesting their potential application in logical reasoning and decision-making tasks and that the autonomous agent learns effective decision-making strategies through iterative training and fine-tuning, demonstrating improved performance in predicting the opponent's secret number. Further research directions include extending this approach to multi-agent settings where multiple autonomous agents can compete or collaborate to refine their strategic reasoning and explore its application in real-world scenarios requiring structured feedback processing.