Notebook-to-Module Refactoring
Experiment Overview
Item
Details
Date 2025-12-27
Goal Extract long functions from notebook cells into reusable Python modules
Environment KINTSUGI Jupyter notebooks, Python modules in notebooks/ directory
Status Success (after fixing)
Context
When notebook cells contain both:
Function definitions (candidates for extraction)Variable definitions (processing parameters, configuration)
Extracting only the functions to a module leaves the variables undefined, breaking downstream cells.
Verified Workflow
CORRECT: Preserve Variable Definitions
When refactoring a cell that contains both functions and variables:
# BEFORE (single notebook cell):
# =============================================================================
# PROCESSING PARAMETERS
# =============================================================================
n_rows = 13
n_cols = 9
start_cycle = 1
end_cycle = 9
n_workers = CPU_COUNT
# =============================================================================
# CHANNEL NAME FUNCTIONS (to be extracted)
# =============================================================================
def load_channel_names(meta_dir):
...
def make_channel_names_unique(channel_dict):
...
# Usage
channel_name_dict = load_channel_names(meta_dir)
rows = list(...) # Uses n_rows, n_cols
# AFTER (two notebook cells + module):
# --- Cell 1: Processing Parameters (KEEP IN NOTEBOOK) ---
n_rows = 13
n_cols = 9
start_cycle = 1
end_cycle = 9
n_workers = CPU_COUNT
# --- Cell 2: Module Import + Usage ---
from Kio import load_channel_names, make_channel_names_unique
channel_name_dict = load_channel_names(meta_dir)
rows = list(...) # Still works - variables defined in Cell 1
# --- Kio.py module (NEW FILE) ---
def load_channel_names(meta_dir):
...
def make_channel_names_unique(channel_dict):
...
Checklist Before Refactoring
Failed Attempts (Critical)
Attempt
Why it Failed
Lesson Learned
Moved entire cell content to module
Variables like n_workers, n_rows became undefined
Only move function definitions, keep variables
Replaced cell with just import
Lost 26 variable definitions
Cell content besides functions must be preserved
Edited project folder directly
Changes overwritten on sync
Always edit main repo first (repo-project-sync-workflow)
Variables Commonly Left Behind
When extracting channel name functions, these variables were accidentally removed:
Category
Variables
Grid config
n_rows, n_cols, rows, cols
Processing range
start_cycle, end_cycle, start_channel, end_channel, n_zplanes
HPC settings
n_workers, CPU_COUNT, IO_WORKERS, ZPLANES_PER_GPU, max_cores
Stitching
pou, overlap_percentage, initial_ncc_threshold
BaSiC params
BASIC_IF_DARKFIELD, BASIC_MAX_ITERATIONS, etc.
Blend mode
BLEND_MODE, BLEND_SIGMA
Key Insights
Notebook cells often mix configuration and implementation
Function extraction should be surgical - only the def blocks
Variable definitions are cell-level configuration, not module content
Always verify imports work by running the refactored cells
Use /advise before refactoring to check for related skills
Related Skills
repo-project-sync-workflow - Edit main repo first, then syncchannel-name-parsing - The functions that were extracted
References
KINTSUGI notebooks/Kio.py - Channel name I/O module
KINTSUGI notebooks/2_Cycle_Processing.ipynb - Cell 7 refactoring
1 --- 2 name: notebook-module-refactoring 3 description: Safe refactoring of Jupyter notebook code into Python modules 4 --- 5
6 # Notebook-to-Module Refactoring
7
8 ## Experiment Overview
9 | Item | Details |
10 |------|---------|
11 | **Date** | 2025-12-27 |
12 | **Goal** | Extract long functions from notebook cells into reusable Python modules |
13 | **Environment** | KINTSUGI Jupyter notebooks, Python modules in notebooks/ directory |
14 | **Status** | Success (after fixing) |
15
16 ## Context
17 When notebook cells contain both:
18 1. **Function definitions** (candidates for extraction)
19 2. **Variable definitions** (processing parameters, configuration)
20
21 Extracting only the functions to a module leaves the variables undefined, breaking downstream cells.
22
23 ## Verified Workflow
24
25 ### CORRECT: Preserve Variable Definitions
26 When refactoring a cell that contains both functions and variables:
27
28 ```python
29 # BEFORE (single notebook cell):
30 # =============================================================================
31 # PROCESSING PARAMETERS
32 # =============================================================================
33 n_rows = 13
34 n_cols = 9
35 start_cycle = 1
36 end_cycle = 9
37 n_workers = CPU_COUNT
38
39 # =============================================================================
40 # CHANNEL NAME FUNCTIONS (to be extracted)
41 # =============================================================================
42 def load_channel_names(meta_dir):
43 ...
44
45 def make_channel_names_unique(channel_dict):
46 ...
47
48 # Usage
49 channel_name_dict = load_channel_names(meta_dir)
50 rows = list(...) # Uses n_rows, n_cols
51 ```
52
53 ```python
54 # AFTER (two notebook cells + module):
55
56 # --- Cell 1: Processing Parameters (KEEP IN NOTEBOOK) ---
57 n_rows = 13
58 n_cols = 9
59 start_cycle = 1
60 end_cycle = 9
61 n_workers = CPU_COUNT
62
63 # --- Cell 2: Module Import + Usage ---
64 from Kio import load_channel_names, make_channel_names_unique
65
66 channel_name_dict = load_channel_names(meta_dir)
67 rows = list(...) # Still works - variables defined in Cell 1
68
69 # --- Kio.py module (NEW FILE) ---
70 def load_channel_names(meta_dir):
71 ...
72 def make_channel_names_unique(channel_dict):
73 ...
74 ```
75
76 ### Checklist Before Refactoring
77 1. [ ] Identify ALL variable definitions in the cell
78 2. [ ] Identify ALL function definitions to extract
79 3. [ ] Check what the remaining code (after function removal) depends on
80 4. [ ] Create module with ONLY the functions
81 5. [ ] Keep variable definitions in notebook cell
82 6. [ ] Add import statement to notebook
83 7. [ ] Test that all downstream cells still work
84
85 ## Failed Attempts (Critical)
86
87 | Attempt | Why it Failed | Lesson Learned |
88 |---------|---------------|----------------|
89 | Moved entire cell content to module | Variables like `n_workers`, `n_rows` became undefined | Only move function definitions, keep variables |
90 | Replaced cell with just import | Lost 26 variable definitions | Cell content besides functions must be preserved |
91 | Edited project folder directly | Changes overwritten on sync | Always edit main repo first (repo-project-sync-workflow) |
92
93 ## Variables Commonly Left Behind
94
95 When extracting channel name functions, these variables were accidentally removed:
96
97 | Category | Variables |
98 |----------|-----------|
99 | Grid config | `n_rows`, `n_cols`, `rows`, `cols` |
100 | Processing range | `start_cycle`, `end_cycle`, `start_channel`, `end_channel`, `n_zplanes` |
101 | HPC settings | `n_workers`, `CPU_COUNT`, `IO_WORKERS`, `ZPLANES_PER_GPU`, `max_cores` |
102 | Stitching | `pou`, `overlap_percentage`, `initial_ncc_threshold` |
103 | BaSiC params | `BASIC_IF_DARKFIELD`, `BASIC_MAX_ITERATIONS`, etc. |
104 | Blend mode | `BLEND_MODE`, `BLEND_SIGMA` |
105
106 ## Key Insights
107 - Notebook cells often mix configuration and implementation
108 - Function extraction should be surgical - only the `def` blocks
109 - Variable definitions are cell-level configuration, not module content
110 - Always verify imports work by running the refactored cells
111 - Use `/advise` before refactoring to check for related skills
112
113 ## Related Skills
114 - `repo-project-sync-workflow` - Edit main repo first, then sync
115 - `channel-name-parsing` - The functions that were extracted
116
117 ## References
118 - KINTSUGI `notebooks/Kio.py` - Channel name I/O module
119 - KINTSUGI `notebooks/2_Cycle_Processing.ipynb` - Cell 7 refactoring
smith6jt-cop