# J.A.R.V.I.S > JARVIS AI Skills Manifest - Skill: `aly-joseph/j-a-r-v-i-s` (Agent Skill, multi-file: 55 files) - Install (CLI): `npx skillmds@latest add aly-joseph/j-a-r-v-i-s` - Raw SKILL.md: https://api.skillmd.com/api/skills/aly-joseph/j-a-r-v-i-s/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: Aly-Joseph (https://skillmd.com/u/aly-joseph) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/aly-joseph/j-a-r-v-i-s --- # JARVIS AI Skills Manifest ## Slug `jarvis-skills` ## Overview Complete skill set for JARVIS AI - an Iron Man-inspired AI desktop assistant with robotic control capabilities. ## Skills Included ### 1. Voice Recognition - **Slug**: `voice-recognition` - **Version**: 1.0.0 - **Description**: Real-time voice input capture with speech-to-text conversion - **Module**: `voice.py` - **Dependencies**: SpeechRecognition, PyAudio, numpy - **Capabilities**: Voice input, speech recognition, multi-language support, noise filtering ### 2. Vision Automation - **Slug**: `vision-automation` - **Version**: 1.0.0 - **Description**: Screen reading, object detection, and automated clicking - **Modules**: `vision_engine.py`, `vision_ocr.py`, `vision_click.py` - **Dependencies**: OpenCV, Tesseract-OCR, PyAutoGUI, Pillow - **Capabilities**: Screen reading, OCR, UI detection, click automation ### 3. Robotic Control (OpenClaw) - **Slug**: `robotic-control` - **Version**: 2.0.0 - **Description**: OpenClaw integration for robotic arm and gripper manipulation - **Module**: `openclaw_control.py` - **Dependencies**: openclaw, pyserial, numpy - **Capabilities**: Robotic arm control, gripper manipulation, precise positioning, force sensing, collision detection - **Features**: - 6-DOF robotic arm movement - Grab/release operations - Precise positioning (x, y, z) - Rotation control - Force/torque sensing - Collision detection - Action sequences - Hardware auto-detection - Simulation mode support ### 4. AI Generation - **Slug**: `ai-generation` - **Version**: 1.0.0 - **Description**: GPT-powered code and content generation with context awareness - **Modules**: `universal_generator.py`, `dynamic_write_helper.py`, `brain.py` - **Dependencies**: requests, python-dotenv - **Capabilities**: Code generation, content writing, context awareness, multi-language support ### 5. Memory System - **Slug**: `memory-system` - **Version**: 1.0.0 - **Description**: Persistent JSON-based storage for user data and context - **Modules**: `memory.py`, `message_memory.py`, `continuation_handler.py` - **Dependencies**: json, os, datetime - **Capabilities**: User storage, context persistence, conversation history, preference learning ### 6. Emotion Detection - **Slug**: `emotion-detection` - **Version**: 1.0.0 - **Description**: Emotional state analysis and response adaptation - **Modules**: `emotion_ai.py`, `emotion.py` - **Dependencies**: re, textblob - **Capabilities**: Emotion detection, sentiment analysis, tone adaptation, behavior adjustment ### 7. Task Routing - **Slug**: `task-routing` - **Version**: 1.0.0 - **Description**: Intelligent command routing and task prioritization - **Module**: `router.py` - **Dependencies**: re - **Capabilities**: Intent detection, command routing, task prioritization, disambiguation --- ## Installation ### Prerequisites - Python 3.8+ - Git - Virtual environment (recommended) ### Setup ```bash # Clone repository git clone https://github.com/Aly-Joseph/J.A.R.V.I.S.git cd jarvis-ai # Create virtual environment python -m venv venv venv\Scripts\activate # Windows # Install dependencies pip install -r requirements.txt ``` --- ## Usage ### Voice Commands #### Voice Recognition - "Jarvis, listen" - "Jarvis, repeat that" #### Vision Automation - "Jarvis, what is on screen?" - "Jarvis, click on [element]" #### Robotic Control - "Jarvis, grab the object" - "Jarvis, move to 10 20 30" - "Jarvis, rotate 45 degrees" - "Jarvis, release" - "Jarvis, return to home" - "Jarvis, claw status" #### AI Generation - "Jarvis, generate a Python script" - "Jarvis, create a login page" #### Memory - "Jarvis, remember my name is [name]" - "Jarvis, what do you remember?" #### Task Routing - Routes commands to appropriate handlers - Intelligently interprets user intent --- ## Code Examples ### Voice Recognition ```python from voice import listen command = listen() print(f"You said: {command}") ``` ### Vision Automation ```python from vision_engine import read_screen_text, click_on_text text_lines = read_screen_text() success = click_on_text("Submit Button") ``` ### Robotic Control ```python from openclaw_control import init_claw claw = init_claw() claw.grab(force=50.0) claw.move_to(10, 20, 30) claw.release() ``` ### AI Generation ```python from universal_generator import generate_anything from dynamic_write_helper import get_context content = generate_anything("create a login page", context=get_context()) ``` ### Memory System ```python from memory import remember, recall remember("user_name", "John") name = recall("user_name") ``` ### Emotion Detection ```python from emotion_ai import detect_emotion emotion = detect_emotion("I'm so frustrated!") ``` ### Task Routing ```python from router import route_task task = route_task("generate a Python script") ``` --- ## Architecture ``` Skills → Router → Task Handler → Response ↓ ├─ Voice Recognition (Input) ├─ Vision Automation (Vision) ├─ Robotic Control (Hardware) ├─ AI Generation (Processing) ├─ Memory System (Storage) ├─ Emotion Detection (Analysis) └─ Task Routing (Logic) ``` --- ## Configuration ### Environment Variables (.env) ```env # OpenAI OPENAI_API_KEY=your_key_here # Speech Recognition SPEECH_RECOGNITION_LANG=en-US # OpenClaw OPENCLAW_PORT=COM3 # Windows # OPENCLAW_PORT=/dev/ttyUSB0 # Linux # OPENCLAW_PORT=/dev/tty.usbserial-* # Mac OPENCLAW_BAUDRATE=115200 OPENCLAW_TIMEOUT=5 ``` --- ## Performance Metrics | Skill | Metric | Value | |-------|--------|-------| | Voice Recognition | Latency | <1s | | Voice Recognition | Accuracy | 85-95% | | Vision Automation | Detection Speed | 100-500ms | | Vision Automation | Accuracy | 90%+ | | Robotic Control | Response Time | <10ms | | Robotic Control | Precision | ±0.03-0.1mm | | Robotic Control | Speed | 1-7000 mm/s | | AI Generation | Generation Time | 1-10s | | Memory System | Recall Time | <10ms | | Emotion Detection | Detection Time | <100ms | | Emotion Detection | Accuracy | 80-85% | | Task Routing | Routing Time | <5ms | | Task Routing | Accuracy | 95%+ | --- ## Dependencies ### Core Python - speechrecognition - pyaudio - pyautogui - opencv-python - pytesseract - openclaw - PySide6 - requests - python-dotenv ### Node.js (Optional) - express - socket.io - node-red ### System Requirements - Tesseract-OCR (for vision skills) - OpenClaw SDK (for robotic control) --- ## Hardware Support ### Robotic Arms - Universal Robots (UR) - ABB Robotics - KUKA - Stäubli - Custom embedded systems ### Grippers - Parallel grippers - Adaptive grippers - Specialized end-effectors ### Communication Protocols - USB Serial - Ethernet (TCP/IP) - ROS (Robot Operating System) --- ## Safety Features - Velocity limiting - Joint limits enforcement - Workspace boundary protection - Collision detection with auto-stop - Emergency stop functionality - Force/torque monitoring - Temperature monitoring --- ## Troubleshooting ### Voice Recognition Issues - Check microphone: `python -c "import pyaudio; pyaudio.PyAudio()"` - Verify language setting in `.env` - Check audio levels ### Vision Automation Issues - Verify Tesseract-OCR installation - Check screen resolution compatibility - Ensure required permissions ### Robotic Control Issues - Verify USB/Ethernet connection - Check COM port: `python -m serial.tools.list_ports` - System runs in simulation mode automatically if hardware unavailable - Check workspace limits configuration ### General Issues - Ensure all dependencies installed: `pip install -r requirements.txt` - Check .env configuration - Verify internet connection for web searches --- ## Contributing ### Adding New Skills 1. Create skill folder: `skills/[skill-name]/` 2. Create `SKILL.md` documentation 3. Create `skill.json` metadata 4. Add implementation files 5. Update main `SKILL.md` (this file) 6. Submit pull request ### Guidelines - Use lowercase slugs with dashes only - Include comprehensive documentation - Provide code examples - Test thoroughly - Follow existing code style --- ## Support - **GitHub**: [github.com/Aly-Joseph/J.A.R.V.I.S](https://github.com/Aly-Joseph/J.A.R.V.I.S) - **Issues**: [github.com/Aly-Joseph/J.A.R.V.I.S/issues](https://github.com/Aly-Joseph/J.A.R.V.I.S/issues) - **Email**: 0xjarvisironman@gmail.com --- ## License MIT License - See LICENSE file for details --- ## Author **Aly-Joseph** - GitHub: [github.com/Aly-Joseph](https://github.com/Aly-Joseph) - Email: 0xjarvisironman@gmail.com - Twitter/X: [@AlyJoseph](https://twitter.com/AlyJoseph) --- ## Acknowledgments - Inspired by J.A.R.V.I.S from Marvel's Iron Man - Built with Python open-source community - OpenClaw team for robotic control - ElevenLabs for neural voice synthesis - OpenCV for computer vision --- ## Changelog ### Version 2.0.0 (2026-01-31) - Added OpenClaw integration for robotic control - Improved emotion detection system - Enhanced memory persistence - Better task routing with priority handling - Comprehensive skill documentation ### Version 1.0.0 (2026-01-15) - Initial release - Core voice recognition - Vision automation - AI generation - Memory system - Emotion detection - Task routing --- **Last Updated**: 2026-01-31 **Status**: Active Development **Total Skills**: 7