# Push To Df Trans

> Run the 'push to df_trans' workflow on the remote M3 Ultra machine. This skill should be used when the user asks to push new trading days to df_trans, update transaction data, or run known_good_version.py on the remote server. Handles SSH connection, PYTHONHASHSEED requirements, and sequential day processing.

- Skill: `tankygranny05/push-to-df-trans` (Agent Skill)
- Install (CLI): `npx skillmds@latest add tankygranny05/push-to-df-trans`
- Raw SKILL.md: https://api.skillmd.com/api/skills/tankygranny05/push-to-df-trans/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: tankygranny05 (https://skillmd.com/u/tankygranny05)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/tankygranny05/push-to-df-trans

---


# Push to df_trans (Remote M3 Ultra)
*[Created by Opus: 7c0d2c3a-dd9f-4ce3-a12a-7ac30231a354]*

## Overview

This skill executes the `known_good_version.py` workflow on a remote M3 Ultra machine to push new trading days into the `df_trans_numerical.pickle` file. The workflow labels transactions with hardcoded insider annotations and updates the main transaction database.

## Critical Requirements

### PYTHONHASHSEED=0 is MANDATORY

The remote pickle files were created with deterministic hashing (`PYTHONHASHSEED=0`). Running without this environment variable causes a **KeyError** because hash values for stk/group/type strings won't match the stored hashes.

**Symptom of missing PYTHONHASHSEED:**
```
KeyError: -319328615819148925
```

**Solution:** Always prefix the command with `PYTHONHASHSEED=0`.

### Sequential Execution Required

Each run modifies the same pickle file (`/db23/pickles/df_trans_numerical.pickle`). Days **must be processed sequentially**, never in parallel, to avoid data corruption.

## Connection Details

| Parameter | Value |
|-----------|-------|
| SSH Command | `ssh -p 22222 14.161.37.223` |
| User Alias | `cm3u` (defined in `~/.zshrc`) |
| Python Path | `~/anaconda3/bin/python` |
| Project Dir | `~/PycharmProjects/remote_to_m3_ultra` |
| Script | `workflows/known_good_version.py` |

## File Locations

### Remote Machine (M3 Ultra)

| File | Path |
|------|------|
| Main script | `~/PycharmProjects/remote_to_m3_ultra/workflows/known_good_version.py` |
| Transaction data | `/db23/pickles/df_trans_numerical.pickle` (~10.8 GB) |
| String mappings | `/db23/pickles/dic_s2i_and_dic_i2s.pickle` (~336 MB) |
| New day files | `/db23/df_new_day_numerical/YYYY_MM_DD.pickle` |
| Backup | `/db23/df_trans_backups/df_trans_numerical.pickle` |

### Local Machine

| File | Path |
|------|------|
| Script (synced via PyCharm) | `/Users/sotola/PycharmProjects/remote_to_m3_ultra/workflows/known_good_version.py` |

## Workflow

### Single Day Execution

To process a single day:

```bash
ssh -p 22222 14.161.37.223 "cd ~/PycharmProjects/remote_to_m3_ultra && \
  PYTHONHASHSEED=0 ~/anaconda3/bin/python workflows/known_good_version.py --day YYYY_MM_DD"
```

**Example:**
```bash
ssh -p 22222 14.161.37.223 "cd ~/PycharmProjects/remote_to_m3_ultra && \
  PYTHONHASHSEED=0 ~/anaconda3/bin/python workflows/known_good_version.py --day 2025_12_03"
```

### Multiple Days (Sequential)

When processing multiple days, execute them **one at a time** in chronological order. Wait for each command to complete before starting the next.

**Example for days 2025_12_03 through 2025_12_05:**

```bash
# Day 1
ssh -p 22222 14.161.37.223 "cd ~/PycharmProjects/remote_to_m3_ultra && \
  PYTHONHASHSEED=0 ~/anaconda3/bin/python workflows/known_good_version.py --day 2025_12_03"

# Day 2 (run after Day 1 completes)
ssh -p 22222 14.161.37.223 "cd ~/PycharmProjects/remote_to_m3_ultra && \
  PYTHONHASHSEED=0 ~/anaconda3/bin/python workflows/known_good_version.py --day 2025_12_04"

# Day 3 (run after Day 2 completes)
ssh -p 22222 14.161.37.223 "cd ~/PycharmProjects/remote_to_m3_ultra && \
  PYTHONHASHSEED=0 ~/anaconda3/bin/python workflows/known_good_version.py --day 2025_12_05"
```

### Day Format

Days can be specified in either format (the script normalizes them):
- `2025_12_03` (preferred)
- `20251203`

### Auto-detect Latest Day

To automatically process the latest available day file:

```bash
ssh -p 22222 14.161.37.223 "cd ~/PycharmProjects/remote_to_m3_ultra && \
  PYTHONHASHSEED=0 ~/anaconda3/bin/python workflows/known_good_version.py"
```

(Omit `--day` flag and set `DAY = None` in the script)

## Expected Output

A successful run shows:
1. GlobalVariables initialization with file timestamps
2. Loading df_trans (~2 seconds for 10+ GB)
3. Building label mappings for group, type, insider, type_ah
4. Adding hardcoded labels (50 entries)
5. Pushing the new day
6. Applying mappings and flagging insider trades
7. Writing updated pickle (~2-3 seconds)
8. Total runtime: ~55-60 seconds per day

**Success indicators:**
- `[√] Passed` for sanity checks
- `Added 50 hardcoded label entries`
- Final `write_pickle` for `df_trans_numerical.pickle`

## Syncing Local Changes to Remote

If local script changes need to be synced to remote:

```bash
scp -P 22222 /Users/sotola/PycharmProjects/remote_to_m3_ultra/workflows/known_good_version.py \
  14.161.37.223:~/PycharmProjects/remote_to_m3_ultra/workflows/known_good_version.py
```

## Troubleshooting

### ModuleNotFoundError: No module named 'lib'

The script needs the REPL-friendly import patch at the top. Ensure the local script has:

```python
import sys
from pathlib import Path

try:
    ROOT = Path(__file__).resolve().parents[1]
    for path in (ROOT,):
        path_str = str(path)
        if path_str not in sys.path:
            sys.path.insert(0, path_str)
except Exception:
    """REPL environments don't set __file__."""
    pass
```

Then sync to remote.

### KeyError with large negative number

Missing `PYTHONHASHSEED=0`. Always include it in the command.

### command not found: python

Use the full anaconda path: `~/anaconda3/bin/python` (not `python` or `python3`).