Portfolio risk assessment in Islamic versus conventional banking requires advanced analytical approaches due to complex market dynamics and regulatory differences. Traditional optimization methods often fail to capture non-linear relationships and volatility patterns inherent in these distinct banking sectors. This study introduces an AI-enhanced framework combining Monte Carlo simulation with Solver-based optimization to compare market risks and determine optimal portfolio weights between Islamic and conventional banking stocks. The methodological innovation lies in integrating Microsoft Excel's GRG nonlinear AI solver for real-time portfolio optimization, addressing limitations of conventional Markowitz models. Using 194 daily price observations from LQ45 and JII70 indices (January-October 2024), we analyzed four banking stocks (BRIS, BTPS, BBCA, BBRI) through Monte Carlo VaR simulation with 10,000 iterations and AI-driven optimization at 95% confidence level. Mann-Whitney tests confirmed significant differences between Islamic and conventional banking returns and risk profiles. Results reveal Islamic banking portfolios demonstrate significantly higher risk (VaR: -4.18%, potential loss: IDR 4,180,169) compared to conventional portfolios (VaR: -1.65%, potential loss: IDR 1,652,541). AI optimization yielded distinct allocation strategies: Islamic portfolio (45.4% BRIS, 54.5% BTPS) versus conventional portfolio (74.5% BBCA, 25.4% BBRI). This research provides the first AI-integrated comparative framework for Islamic-conventional banking risk analysis, offering quantitative evidence that challenges assumptions about Islamic banking stability. The study demonstrates practical applications of machine learning in portfolio management for emerging markets, providing actionable insights for different investor risk profiles while advancing methodological approaches in Islamic finance research.