JRVS

Iron Man-Inspired AI Assistant for Gamers & Power Users

View on GitHub

💡 Inspiration

Inspired by Tony Stark's JARVIS from the Iron Man movies, we wanted to create a real-world AI assistant that could actually help gamers and power users with everyday tasks through natural voice commands, complete with a futuristic holographic interface.

🚀 What JRVS Does

JRVS is a voice-controlled AI assistant that combines speech recognition, natural language processing, and visual feedback to provide an Iron Man-style computing experience.
🎙️

Voice Control

Listens for "Jarvis" wake word and processes natural language commands

🤖

AI Brain

Uses Google Gemini AI to understand context and execute complex multi-step actions

🎮

Gaming Automation

Includes AFK macros, chat automation, game launching, and screen clipping integration with OBS

📸

Media Capture

Takes screenshots, webcam photos, and saves gameplay clips on command

🌐

Web Integration

Opens websites, performs web searches, and controls media playback

💻

System Control

Monitors system resources, opens/closes applications, and manages Windows processes

✨

Holographic UI

Electron-based transparent overlay with animated visualizers and Iron Man-inspired aesthetics

🎵

Audio Visualization

Real-time microphone-driven circular visualizers with neon effects

📱

Multi-View Interface

Toggleable between full overlay and minimal modes

🔧 How We Built It

Backend: Python Flask server with Socket.IO for real-time communication, Vosk for offline speech recognition, Google Gemini for AI processing, and ElevenLabs for text-to-speech responses.

Frontend: Electron app creating a transparent always-on-top overlay with HTML5 Canvas animations, Three.js for 3D effects, and WebSocket client for real-time updates.

AI Integration: Custom action mapping system that translates natural language to specific functions, with support for complex gaming macros using PyAutoGUI and pynput for real OS-level input simulation.

Gaming Features: Integration with OBS for clip saving, Steam game detection and launching, CSGO/Valorant chat automation with AI-generated contextual messages.

🏆 Accomplishments

🏆 Seamless Voice Integration
Achieved natural conversation flow with sub-2-second response times from voice command to action execution
🎨 Stunning Visual Design
Created a production-quality Iron Man interface with smooth animations, particle effects, and responsive audio visualizations
🎮 Advanced Gaming Automation
Built a comprehensive macro system that can handle complex gaming scenarios like AFK prevention, chat automation, and contextual AI responses
🔧 Robust Architecture
Developed a modular system that cleanly separates voice processing, AI logic, UI rendering, and system control
⚡ Performance Optimization
Maintained smooth 60fps overlay performance while running continuous speech recognition and system monitoring