Scribbles for my bad memory

Date: September 2, 2025 Context: Investigation and resolution of test failures on Git Bash/Windows

Executive Summary

We encountered widespread test failures when running the eed test suite on Git Bash/Windows, while all tests passed on Linux. Through systematic investigation, we discovered multiple platform-specific issues with bats test framework and developed comprehensive solutions. This document captures the technical journey, root causes, and proven solutions for future reference.

The Problem

Initial Symptoms

• Multiple test failures on Git Bash: test_eed_logging.bats, test_eed_preview.bats, test_eed_single_param.bats, test_eed_stdin.bats
• All tests passed perfectly on Linux
• Mysterious file corruption in safety tests
• Pipeline-based tests consistently failing with "Command not found" errors

Example Failing Patterns

# This pattern consistently failed on Git Bash:
run bash -c "printf '1d\nw\nq\n' | $SCRIPT_UNDER_TEST --force test.txt -"

# Status: 127 (Command not found)
# Error: bash -c printf '1d\nw\nq\n' | /path/to/eed --force test.txt -

Root Cause Analysis

1. Missing Library Dependencies

Issue: eed_common.sh was using EED_REGEX_INPUT_MODE without sourcing eed_regex_patterns.sh

Symptoms:

• Logging tests failed because input mode detection returned empty regex
• Content that should be skipped was being logged

Fix:

# Added to eed_common.sh
source "$(dirname "${BASH_SOURCE[0]}")/eed_regex_patterns.sh"

2. Bats Pipeline Simulation Issues

Issue: Bats implements pipe simulation differently on Windows vs Linux

Technical Details:

• Linux: Native shell pipes or compatible simulation work correctly
• Windows/Git Bash: Bats' pipe parsing breaks complex pipeline commands
• Pattern run bash -c "command | other" becomes bash -c command | other
• The pipeline executes outside bats' control, losing exit code and output capture

Symptoms:

# What we wrote:
run bash -c "printf '1d\nw\nq\n' | $SCRIPT_UNDER_TEST --force test.txt -"

# What actually executed:
bash -c printf '1d\nw\nq\n' | /path/to/eed --force test.txt -
#           ^^^^^^^^^^^^^^^^^^ Only this part in bash -c
#                              ^^^^^^^^^^^^^^^^^^^^^^^^^ This runs outside bats

3. File System Stat Comparison Issues

Issue: stat output includes microsecond-precision access times that change on every file read

Technical Details:

• Tests compared full stat output including access times
• Reading files for verification changed access times
• Caused false failures in file integrity tests

Before:

original_stat="$(stat sample.txt)"
# ... test runs ...
[[ "$(stat sample.txt)" == "$original_stat" ]]  # Always fails due to access time

4. Regex Pattern Compatibility

Issue: Git Bash regex handling differences in substitute command detection

Technical Details:

• Fallback regex was too restrictive: s(.)[^\\]*\1.*\1([0-9gp]+)?$
• Pattern [^\\]* excluded characters needed for patterns like console\.log
• Made regex more permissive while maintaining safety

Solutions Implemented

Solution 1: Fix Library Dependencies

# In lib/eed_common.sh - added missing source
source "$(dirname "${BASH_SOURCE[0]}")/eed_regex_patterns.sh"

Solution 2: Cross-Platform Pipeline Patterns

A. Heredoc Approach (Recommended for Complex Input)

# Before (fails on Git Bash):
run bash -c "printf '1c\nchanged\n.\nw\nq\n' | $SCRIPT_UNDER_TEST --force test.txt -"

# After (works everywhere):
run "$SCRIPT_UNDER_TEST" --force test.txt - << 'EOF'
1c
changed
.
w
q
EOF

B. GPT's Pipeline-in-Bash-C Pattern (For When Pipes Are Needed)

# Before (fails on Git Bash):
run bash -c "echo '$script' | '$SCRIPT_UNDER_TEST' --force '$TEST_FILE' -"

# After (works everywhere):
run bash -c 'set -o pipefail; echo "$1" | "$2" --force "$3" -' \
    bash "$script" "$SCRIPT_UNDER_TEST" "$TEST_FILE"

Key Elements:

• Single quotes around entire bash -c content
• set -o pipefail for proper error propagation
• Pass variables as arguments ("$1", "$2") to avoid quoting hell
• Entire pipeline contained within one bash -c execution

Solution 3: Robust File Integrity Testing

# Before (fails due to access time changes):
original_stat="$(stat sample.txt)"
[[ "$(stat sample.txt)" == "$original_stat" ]]

# After (only check relevant attributes):
original_size="$(stat -c %s sample.txt)"
original_mtime="$(stat -c %Y sample.txt)"
original_inode="$(stat -c %i sample.txt)"

[[ "$(stat -c %s sample.txt)" == "$original_size" ]]     # Size unchanged
[[ "$(stat -c %Y sample.txt)" == "$original_mtime" ]]   # Modify time unchanged
[[ "$(stat -c %i sample.txt)" == "$original_inode" ]]   # Inode unchanged

Solution 4: Improved Regex Patterns

# Before (too restrictive):
fallback='s(.)[^\\]*\1.*\1([0-9gp]+)?$'

# After (handles escaped characters properly):
fallback='s([^[:space:]]).*\1.*\1([0-9gp]*)?$'

Testing and Validation

Proof-of-Concept Tests

We created tests/test_printf_pipeline.bats to validate our understanding:

1. Direct printf pipelines work perfectly (bypassing bats)
2. GPT's approach works reliably (pipeline within bash -c)
3. Problematic patterns consistently fail (pipeline across bash -c boundary)

Results

• Before: Multiple test failures, warnings, file corruption fears
• After: 256 tests pass, 0 failures, 1 expected skip, 0 warnings

Key Learnings

1. Platform-Specific Tool Behavior

• Never assume cross-platform tools work identically
• Bats, while excellent, has platform-specific implementation differences
• Always test on target platforms, not just development environment

2. Root Cause Investigation Methodology

• Don't guess, investigate systematically
• Use bats -x for detailed execution traces
• Test hypotheses with isolated proof-of-concept code
• Distinguish between symptoms and root causes

3. Regex and Shell Compatibility

• Git Bash supports modern regex features when used correctly
• Issues often stem from tooling layer, not shell capabilities
• Platform differences in command parsing require careful attention

4. Test Design Best Practices

• Avoid external dependencies in tests (like python3 for JSON validation)
• Use heredoc for complex multiline input - most reliable approach
• Compare only stable file attributes - avoid access times
• Separate concerns - one test per scenario for better debugging

Recommended Patterns for Future Development

✅ DO: Use Heredoc for Complex Input

run "$COMMAND" file.txt - << 'EOF'
multiline
script
content
EOF

✅ DO: GPT's Pattern for Necessary Pipelines

run bash -c 'set -o pipefail; echo "$1" | "$2" --flags "$3"' \
    bash "$input" "$command" "$target"

❌ DON'T: Pipeline Across bash -c Boundary

run bash -c "printf '...' | command ..."  # Breaks on Git Bash

❌ DON'T: Compare Volatile File Attributes

[[ "$(stat file.txt)" == "$original_stat" ]]  # Access time changes

Files Modified

Core Library

• lib/eed_common.sh: Added missing regex patterns source
• lib/eed_regex_patterns.sh: Improved substitute regex fallback

Test Files

• tests/test_eed_single_param.bats: Printf pipeline → heredoc
• tests/test_eed_stdin.bats: Printf pipeline → heredoc + GPT pattern
• tests/test_safety_override_integration.bats: All patterns → GPT approach
• tests/test_ai_file_lifecycle.bats: Removed python3 dependency
• tests/test_eed_preview.bats: Fixed stat comparison + separated safety tests

New Infrastructure

• tests/test_printf_pipeline.bats: Comprehensive pipeline pattern validation

Impact and Metrics

• Test Reliability: 256/256 tests now pass consistently on Git Bash
• Warning Elimination: 0 BW01 warnings (previously multiple)
• Cross-Platform Compatibility: Patterns work on both Windows and Linux
• Maintainability: Cleaner test patterns, better separation of concerns
• Documentation: Comprehensive understanding of platform differences

Future Considerations

When Adding New Tests

1. Use heredoc approach for complex multiline input
2. Apply GPT's pattern when pipelines are absolutely necessary
3. Avoid comparing volatile file system attributes
4. Test on both platforms before considering complete

When Debugging Cross-Platform Issues

1. Use bats -x to see exact command execution
2. Create isolated test cases to verify hypotheses
3. Check for tool-specific implementation differences
4. Don't assume the issue is with your code - could be tooling

Monitoring

• Watch for new BW01 warnings as indicator of problematic patterns
• Ensure CI/CD tests both Linux and Windows environments
• Regular cross-platform test execution during development

This investigation demonstrates the importance of thorough cross-platform testing and systematic root cause analysis. The solutions we implemented not only fixed immediate issues but established robust patterns for future development.

Key Takeaway: When tools behave differently across platforms, the solution isn't to work around the differences, but to understand them deeply and adopt patterns that work reliably everywhere.