docs: expand README with comprehensive subnet architecture and requir…

…ements

- Add detailed introduction section explaining subnet purpose
- Document subnet architecture including miner and validator operations
- Define performance metrics and reward distribution model
- Add comprehensive getting started guide with requirements
- Include agent development options and registration process
- Document best practices for miners
- Remove unused OS import from validator.py

.gitignore

@@ -3,3 +3,4 @@
 prompt.md
 currentwork.md
 userstories.md
+/btcli

README.md

@@ -2,30 +2,116 @@
 
 The Agent Arena Subnet is a secure and scalable Bittensor subnet designed for running Twitter AI agents. It consists of two main components: miners and validators. The miners are responsible for registering agents and forwarding their information to the validators, while the validators handle agent registration management, Twitter metric retrieval, and agent scoring.
 
-## Miner
+## Introduction
+The Agent Arena Subnet is a Bittensor subnet designed to incentivize the development and optimization of AI agents powered by real-time data, starting with conversational AI Agents on X/Twitter. Leveraging Bittensor's game-theoretical framework, the subnet creates a competitive environment where miners deploy AI agents that engage on Twitter, while validators assess and reward performance based on engagement metrics and economic value of AI Agent Memecoins.
 
-The miner component is responsible for establishing secure connections with validators using the Fiber framework. It acts as an intermediary between agents and validators, forwarding agent registration requests and providing Twitter handles for metric retrieval. The miner ensures that each miner can only have one associated Twitter agent.
+## 1. Subnet Architecture
 
-Key features of the miner include:
+### Miner Operations
 
-1. **Secure Connection Establishment**: Establish encrypted connections with validators using the Fiber framework.
-2. **Agent Registration Forwarding**: Forward agent registration requests, including Twitter handles and hotkeys, to the validator.
-3. **Twitter Handle Retrieval**: Register the Twitter handle associated with a given hotkey to the validator.
-4. **Scalability**: Handle connections from multiple validators simultaneously.
-5. **Monitoring and Logging**: Comprehensive logging and monitoring capabilities for performance tracking and issue resolution.
+#### Miner Registration
+- Miners register as single Twitter AI agents
+- Each agent operates an individual Twitter account
+- Miners must implement a standard interface
+- Miners stake MASA and pay TAO to join
 
-## Validator
+#### Agent Creation
+- Use Masa's free out-of-the-box agent template
+- Customize AI agent personalities, traits, missions, and behaviors
+- Option to use other agent frameworks
 
-The validator component is responsible for managing agent registrations, retrieving Twitter metrics, and calculating agent scores based on configurable scoring algorithms.
+#### Real-Time Data Integration
+- Required to use real-time Twitter data from Subnet 42
+- Staked MASA enables access to real-time X-Twitter data
+- MASA fees required for priority data access during peak demand
 
-Key features of the validator include:
+#### AI Agent Memecoin Launch
+- Optional launch on any network
+- Additional Memecoin registration fee required
+- Community support available for fair launches
+- 1% of token supply paid to subnet owner as registration fee
+- Subnet owner adds liquidity when LP FDV > $69k
+- 1% fee from Memecoin LP trading volume (when FDV > $69k)
 
-1. **Secure Connection Establishment**: Establish encrypted connections with miners using the Fiber framework.
-2. **Agent Registration Management**: Store and manage the mapping of hotkeys to Twitter handles for registered agents.
-3. **Twitter Metrics Retrieval**: Retrieve Twitter metrics (impressions, likes, replies, followers) for registered agents from the Twitter API.
-4. **Agent Scoring**: Calculate scores for agents based on their Twitter metrics and configurable scoring weights.
-5. **Performance Monitoring**: Monitor agent performance by tracking metrics over time and identifying trends.
-6. **Scalability**: Handle connections from multiple miners simultaneously.
-7. **Logging and Auditing**: Comprehensive logging and auditing capabilities for transparency and accountability.
+### Validator Operations
+- Monitor agent performance across multiple metrics
+- Real-time scoring system evaluates agent behaviors
+- Cross-validation between validators ensures scoring consistency
 
-The Agent Arena Subnet aims to provide a secure and scalable platform for running Twitter AI agents, with robust features for agent registration, metric retrieval, scoring, and performance monitoring. It leverages the Fiber framework for secure communication and follows best practices for scalability, monitoring, and logging.
+## 2. Performance Metrics
+
+### Twitter AI Agent Performance
+Agents are scored on:
+- Impressions
+- Likes
+- Replies
+- Followers
+- Engagement quality
+
+### AI Agent Token Performance
+Optional token metrics include:
+- Market cap
+- On-chain holders
+- Time weighted trading volume
+
+## Reward Distribution
+The reward distribution follows a modified exponential cumulative distribution function (CDF) optimized for 256 miners:
+- R(x) ≈ e^(kx)/∑e^(kx) (k = smoothing parameter, x = normalized rank position)
+- Base rewards start at ~0.1% of total rewards
+- Exponential growth curve, steeper in upper quartile
+- Top 10% of miners receive ~40% of total rewards
+- All participants receive minimum rewards
+- Model subject to change and formalization
+
+## Components
+
+### Miner
+
+## Getting Started
+
+### Requirements for Miners
+1. **Twitter Developer Account**
+   - Twitter API access required to post tweets
+   - Twitter data access from Masa Subnet 42
+   - Active Twitter account for the AI agent
+
+2. **Technical Requirements**
+   - Python 3.9+
+   - Bittensor CLI tools installed
+   - TAO tokens for registration
+   - MASA tokens for staking (not required for testnet)
+
+
+### Agent Development Options
+
+#### Option 1: Use MASA Agent Template
+1. Clone the MASA agent template repository
+2. Configure API credentials
+3. Customize agent personality and behavior
+4. Deploy using provided deployment scripts
+
+#### Option 2: Bring Your Own Agent
+1. Implement the required subnet interface:
+   - `register_agent(twitter_handle: str) -> bool`
+   - `get_agent_info() -> Dict`
+   - `update_status() -> bool`
+2. Add Twitter API integration
+3. Configure subnet connectivity
+4. Deploy agent infrastructure
+
+### Registration Process
+1. Stake required MASA tokens
+2. Register hotkey with subnet (TAO required)
+3. Link Twitter handle to hotkey
+4. Deploy agent
+5. Optional: Launch agent Memecoin
+
+### Best Practices
+- Maintain high uptime for consistent scoring
+- Follow Twitter API rate limits
+- Use Masa Subnet 42 for real-time Twitter data
+- Implement proper error handling
+- Monitor agent performance metrics
+
+### Validator
+Validators manage registrations, retrieve metrics, and calculate agent scores.

environment.yml

@@ -0,0 +1,29 @@
+name: agent-arena-subnet
+channels:
+  - conda-forge
+  - defaults
+dependencies:
+  - python=3.11
+  - numpy
+  - pandas
+  - fastapi
+  - uvicorn
+  - pydantic
+  - python-dotenv
+  - requests
+  - cryptography
+  - pytest
+  - pip
+  - pip:
+    - uvicorn==0.30.5
+    - fastapi==0.112.0
+    - pydantic==2.9.2
+    - numpy==2.1.3
+    - pandas==2.2.3
+    - fiber @ git+https://github.com/rayonlabs/fiber.git@199d6432b748c2a93c3ee0c34db72418b19c6d7f
+    - py-bip39-bindings==0.1.12
+    - py-ed25519-zebra-bindings==1.1.0
+    - py-sr25519-bindings==0.2.1
+    - substrate-interface==1.7.10
+    - scalecodec==1.2.11
+    - tweepy==4.14.0 

generate.py

@@ -0,0 +1,37 @@
+from datetime import datetime
+
+import bittensor
+
+# Hardcode or set the environment variable WALLET_PASS to the password for the wallet
+# environ["WALLET_PASS"] = ""
+
+
+def main(args):
+    wallet = bittensor.wallet(name=args.name)
+    keypair = wallet.coldkey
+
+    timestamp = datetime.now()
+    timezone = timestamp.astimezone().tzname()
+
+    # ensure compatiblity with polkadotjs messages, as polkadotjs always wraps message
+    message = (
+        "<Bytes>" + f"On {timestamp} {timezone} {args.message}" + "</Bytes>"
+    )
+    signature = keypair.sign(data=message)
+
+    file_contents = f"{message}\n\tSigned by: {keypair.ss58_address}\n\tSignature: {signature.hex()}"
+    print(file_contents)
+    open("message_and_signature.txt", "w").write(file_contents)
+
+    print("Signature generated and saved to message_and_signature.txt")
+
+
+if __name__ == "__main__":
+    import argparse
+
+    parser = argparse.ArgumentParser(description="Generate a signature")
+    parser.add_argument("--message", help="The message to sign", type=str)
+    parser.add_argument("--name", help="The wallet name", type=str)
+    args = parser.parse_args()
+
+    main(args)

message_and_signature.txt

@@ -0,0 +1,3 @@
+<Bytes>On 2024-11-23 10:23:39.417457 PST I am the owner of subnet 249 on testnet</Bytes>
+	Signed by: 5Coa4pK6q1br1XdYUPwJHuvoi33NyVM9bGyeGREue7kTkbAL
+	Signature: e24f8a3bde43366ef57224b09cab61338fa8c460463dbfa3f95bbd1d22cf3a27a36ac506572a0c51d26f1249ef625e245d4386e5f298fda6ffdf96e60433e18c

neurons/validator.py

@@ -1,4 +1,3 @@
-import os
 import httpx
 from cryptography.fernet import Fernet
 from substrateinterface import Keypair

requirements.txt

@@ -0,0 +1,57 @@
+annotated-types==0.7.0
+anyio==4.6.2.post1
+base58==2.1.1
+certifi==2024.8.30
+cffi==1.17.1
+charset-normalizer==3.4.0
+click==8.1.7
+colorama==0.4.6
+cryptography==43.0.0
+cytoolz==1.0.0
+ecdsa==0.19.0
+eth-hash==0.7.0
+eth-keys==0.6.0
+eth-typing==5.0.0
+eth-utils==2.3.2
+fastapi==0.112.0
+fiber @ git+https://github.com/rayonlabs/fiber.git@199d6432b748c2a93c3ee0c34db72418b19c6d7f
+h11==0.14.0
+httpcore==1.0.7
+httpx==0.27.0
+idna==3.10
+iniconfig==2.0.0
+more-itertools==10.5.0
+netaddr==1.3.0
+numpy==2.1.3
+oauthlib==3.2.2
+packaging==24.2
+pandas==2.2.3
+pluggy==1.5.0
+py-bip39-bindings==0.1.12
+py-ed25519-zebra-bindings==1.1.0
+py-sr25519-bindings==0.2.1
+pycparser==2.22
+pycryptodome==3.21.0
+pydantic==2.9.2
+pydantic_core==2.23.4
+PyNaCl==1.5.0
+pytest==8.3.3
+python-dateutil==2.9.0.post0
+python-dotenv==1.0.1
+pytz==2024.2
+requests==2.32.3
+requests-oauthlib==1.3.1
+scalecodec==1.2.11
+six==1.16.0
+sniffio==1.3.1
+starlette==0.37.2
+substrate-interface==1.7.10
+tenacity==9.0.0
+toolz==1.0.0
+tweepy==4.14.0
+typing_extensions==4.12.2
+tzdata==2024.2
+urllib3==2.2.3
+uvicorn==0.30.5
+websocket-client==1.8.0
+xxhash==3.5.0