MAF-RL: Multi-Source Actor-Critic fusion reinforcement learning for dynamic decision systems

초록

Dynamic decision systems increasingly depend on the integration of heterogeneous information sources-textual, visual, contextual, and relational-to achieve adaptive and context-aware intelligence. However, existing learning frameworks often process these signals in isolation, limiting their ability to adapt decisions over time. To address this challenge, we propose MAF-RL, a Multi-Source Actor-Critic Fusion Reinforcement Learning (RL) framework that formulates sequential recommendation primarily as an RL problem and uses multi-source fusion to construct expressive state representations for the agent. The principal novelty of MAF-RL lies not in the fusion operator itself, but in its role as a decision-aware state construction mechanism. Diverse data streams-including sequential histories, textual semantics, visual representations, contextual metadata, and relational signals-are integrated into a unified RL state that is optimized end-toend through long-horizon Actor-Critic policy learning rather than short-term prediction loss. The Actor-Critic architecture, optimized through Proximal Policy Optimization (PPO), learns dynamic policies guided by a multi-objective reward that balances immediate performance, novelty, and strategic repetition. By grounding policy learning on fused multi-source states, this formulation enables the agent to reason over multi-source evidence and adapt actions across evolving envi-ronments. Empirical evaluation on three large-scale multi-source benchmarks -MovieLens-1 M, Amazon-Books, and Yelp -demonstrates that MAF-RL consistently outperforms state-of-the-art baselines, achieving superior ranking accuracy (HR@10, NDCG@10) and a better trade-off be-tween repetition and novelty (RR@10, Novelty@10). Overall, MAF-RL should be viewed as an RL-based sequential decision framework whose effectiveness derives from its multi-source state construction, enabling more adaptive and principled behavior in dynamic recommendation settings.

키워드