Date: September 2, 2025
Context: Investigation and resolution of test failures on Git Bash/Windows
Executive Summary
We encountered widespread test failures when running the eed test suite on Git Bash/Windows, while all tests passed on Linux. Through systematic investigation, we discovered multiple platform-specific issues with bats test framework and developed comprehensive solutions. This document captures the technical journey, root causes, and proven solutions for future reference.
The Problem
Initial Symptoms
- Multiple test failures on Git Bash:
test_eed_logging.bats, test_eed_preview.bats, test_eed_single_param.bats, test_eed_stdin.bats
- All tests passed perfectly on Linux
- Mysterious file corruption in safety tests
- Pipeline-based tests consistently failing with "Command not found" errors
Example Failing Patterns
# This pattern consistently failed on Git Bash:
run bash -c "printf '1d\nw\nq\n' | $SCRIPT_UNDER_TEST --force test.txt -"
# Status: 127 (Command not found)
# Error: bash -c printf '1d\nw\nq\n' | /path/to/eed --force test.txt -
Root Cause Analysis
1. Missing Library Dependencies
Issue: eed_common.sh was using EED_REGEX_INPUT_MODE without sourcing eed_regex_patterns.sh
Symptoms:
- Logging tests failed because input mode detection returned empty regex
- Content that should be skipped was being logged
Fix:
# Added to eed_common.sh
source "$(dirname "${BASH_SOURCE[0]}")/eed_regex_patterns.sh"
2. Bats Pipeline Simulation Issues
Issue: Bats implements pipe simulation differently on Windows vs Linux
Technical Details:
- Linux: Native shell pipes or compatible simulation work correctly
- Windows/Git Bash: Bats' pipe parsing breaks complex pipeline commands
- Pattern
run bash -c "command | other" becomes bash -c command | other
- The pipeline executes outside bats' control, losing exit code and output capture
Symptoms:
# What we wrote:
run bash -c "printf '1d\nw\nq\n' | $SCRIPT_UNDER_TEST --force test.txt -"
# What actually executed:
bash -c printf '1d\nw\nq\n' | /path/to/eed --force test.txt -
# ^^^^^^^^^^^^^^^^^^ Only this part in bash -c
# ^^^^^^^^^^^^^^^^^^^^^^^^^ This runs outside bats
3. File System Stat Comparison Issues
Issue: stat output includes microsecond-precision access times that change on every file read
Technical Details:
- Tests compared full
stat output including access times
- Reading files for verification changed access times
- Caused false failures in file integrity tests
Before:
original_stat="$(stat sample.txt)"
# ... test runs ...
[[ "$(stat sample.txt)" == "$original_stat" ]] # Always fails due to access time
4. Regex Pattern Compatibility
Issue: Git Bash regex handling differences in substitute command detection
Technical Details:
- Fallback regex was too restrictive:
s(.)[^\\]*\1.*\1([0-9gp]+)?$
- Pattern
[^\\]* excluded characters needed for patterns like console\.log
- Made regex more permissive while maintaining safety
Solutions Implemented
Solution 1: Fix Library Dependencies
# In lib/eed_common.sh - added missing source
source "$(dirname "${BASH_SOURCE[0]}")/eed_regex_patterns.sh"
Solution 2: Cross-Platform Pipeline Patterns
A. Heredoc Approach (Recommended for Complex Input)
# Before (fails on Git Bash):
run bash -c "printf '1c\nchanged\n.\nw\nq\n' | $SCRIPT_UNDER_TEST --force test.txt -"
# After (works everywhere):
run "$SCRIPT_UNDER_TEST" --force test.txt - << 'EOF'
1c
changed
.
w
q
EOF
B. GPT's Pipeline-in-Bash-C Pattern (For When Pipes Are Needed)
# Before (fails on Git Bash):
run bash -c "echo '$script' | '$SCRIPT_UNDER_TEST' --force '$TEST_FILE' -"
# After (works everywhere):
run bash -c 'set -o pipefail; echo "$1" | "$2" --force "$3" -' \
bash "$script" "$SCRIPT_UNDER_TEST" "$TEST_FILE"
Key Elements:
- Single quotes around entire bash -c content
set -o pipefail for proper error propagation- Pass variables as arguments (
"$1", "$2") to avoid quoting hell
- Entire pipeline contained within one bash -c execution
Solution 3: Robust File Integrity Testing
# Before (fails due to access time changes):
original_stat="$(stat sample.txt)"
[[ "$(stat sample.txt)" == "$original_stat" ]]
# After (only check relevant attributes):
original_size="$(stat -c %s sample.txt)"
original_mtime="$(stat -c %Y sample.txt)"
original_inode="$(stat -c %i sample.txt)"
[[ "$(stat -c %s sample.txt)" == "$original_size" ]] # Size unchanged
[[ "$(stat -c %Y sample.txt)" == "$original_mtime" ]] # Modify time unchanged
[[ "$(stat -c %i sample.txt)" == "$original_inode" ]] # Inode unchanged
Solution 4: Improved Regex Patterns
# Before (too restrictive):
fallback='s(.)[^\\]*\1.*\1([0-9gp]+)?$'
# After (handles escaped characters properly):
fallback='s([^[:space:]]).*\1.*\1([0-9gp]*)?$'
Testing and Validation
Proof-of-Concept Tests
We created tests/test_printf_pipeline.bats to validate our understanding:
- Direct printf pipelines work perfectly (bypassing bats)
- GPT's approach works reliably (pipeline within bash -c)
- Problematic patterns consistently fail (pipeline across bash -c boundary)
Results
- Before: Multiple test failures, warnings, file corruption fears
- After: 256 tests pass, 0 failures, 1 expected skip, 0 warnings
Key Learnings
1. Platform-Specific Tool Behavior
- Never assume cross-platform tools work identically
- Bats, while excellent, has platform-specific implementation differences
- Always test on target platforms, not just development environment
2. Root Cause Investigation Methodology
- Don't guess, investigate systematically
- Use
bats -x for detailed execution traces
- Test hypotheses with isolated proof-of-concept code
- Distinguish between symptoms and root causes
3. Regex and Shell Compatibility
- Git Bash supports modern regex features when used correctly
- Issues often stem from tooling layer, not shell capabilities
- Platform differences in command parsing require careful attention
4. Test Design Best Practices
- Avoid external dependencies in tests (like
python3 for JSON validation)
- Use heredoc for complex multiline input - most reliable approach
- Compare only stable file attributes - avoid access times
- Separate concerns - one test per scenario for better debugging
Recommended Patterns for Future Development
✅ DO: Use Heredoc for Complex Input
run "$COMMAND" file.txt - << 'EOF'
multiline
script
content
EOF
✅ DO: GPT's Pattern for Necessary Pipelines
run bash -c 'set -o pipefail; echo "$1" | "$2" --flags "$3"' \
bash "$input" "$command" "$target"
❌ DON'T: Pipeline Across bash -c Boundary
run bash -c "printf '...' | command ..." # Breaks on Git Bash
❌ DON'T: Compare Volatile File Attributes
[[ "$(stat file.txt)" == "$original_stat" ]] # Access time changes
Files Modified
Core Library
lib/eed_common.sh: Added missing regex patterns sourcelib/eed_regex_patterns.sh: Improved substitute regex fallback
Test Files
tests/test_eed_single_param.bats: Printf pipeline → heredoctests/test_eed_stdin.bats: Printf pipeline → heredoc + GPT patterntests/test_safety_override_integration.bats: All patterns → GPT approachtests/test_ai_file_lifecycle.bats: Removed python3 dependencytests/test_eed_preview.bats: Fixed stat comparison + separated safety tests
New Infrastructure
tests/test_printf_pipeline.bats: Comprehensive pipeline pattern validation
Impact and Metrics
- Test Reliability: 256/256 tests now pass consistently on Git Bash
- Warning Elimination: 0 BW01 warnings (previously multiple)
- Cross-Platform Compatibility: Patterns work on both Windows and Linux
- Maintainability: Cleaner test patterns, better separation of concerns
- Documentation: Comprehensive understanding of platform differences
Future Considerations
When Adding New Tests
- Use heredoc approach for complex multiline input
- Apply GPT's pattern when pipelines are absolutely necessary
- Avoid comparing volatile file system attributes
- Test on both platforms before considering complete
When Debugging Cross-Platform Issues
- Use
bats -x to see exact command execution
- Create isolated test cases to verify hypotheses
- Check for tool-specific implementation differences
- Don't assume the issue is with your code - could be tooling
Monitoring
- Watch for new BW01 warnings as indicator of problematic patterns
- Ensure CI/CD tests both Linux and Windows environments
- Regular cross-platform test execution during development
This investigation demonstrates the importance of thorough cross-platform testing and systematic root cause analysis. The solutions we implemented not only fixed immediate issues but established robust patterns for future development.
Key Takeaway: When tools behave differently across platforms, the solution isn't to work around the differences, but to understand them deeply and adopt patterns that work reliably everywhere.
