Automated Directory Hygiene

Comprehensive Project Structure Maintenance & Optimization

Task Overview

Maintain pristine project structures through continuous monitoring, intelligent cleanup, and proactive optimization. This task transforms reactive directory maintenance into a proactive system that prevents organizational decay and ensures long-term project health through automated hygiene management.

Core Objectives

1. Monitor directory health through continuous structural and content analysis

2. Detect and resolve anomalies using intelligent pattern recognition and rule-based systems

3. Optimize storage efficiency through deduplication, compression, and archival strategies

4. Generate actionable insights through comprehensive reporting and trend analysis

5. Maintain system performance while ensuring zero disruption to active workflows

Input Requirements

Hygiene Scan Configuration

hygiene_scan_config:
  scan_scope:
    - target_directories: "list_of_directories_to_monitor"
    - exclusion_patterns: "patterns_to_exclude_from_scanning"
    - depth_limit: "maximum_directory_traversal_depth"
    - file_size_limits: "minimum_and_maximum_file_sizes_to_process"
  
  scan_frequency:
    - continuous_monitoring: "real_time_critical_issues"
    - daily_maintenance: "routine_cleanup_operations"
    - weekly_deep_scan: "comprehensive_analysis"
    - monthly_optimization: "strategic_improvements"
  
  analysis_parameters:
    - duplicate_detection_threshold: "minimum_file_size_for_duplicate_checking"
    - empty_directory_grace_period: "days_before_flagging_empty_directories"
    - obsolete_file_criteria: "age_and_access_patterns_for_obsolete_files"
    - permission_consistency_rules: "expected_permission_patterns"

Execution Workflow

Phase 1: Comprehensive Directory Scanning (2-10 minutes)

def perform_comprehensive_directory_scan(scan_config):
    """
    Multi-dimensional analysis of project directory structure
    """
    scan_results = {
        'structural_analysis': {
            'directory_tree': build_complete_directory_tree(scan_config.target_directories),
            'empty_directories': identify_empty_directories(scan_config),
            'orphaned_files': find_files_without_logical_placement(scan_config),
            'broken_symlinks': detect_broken_symbolic_links(scan_config),
            'permission_anomalies': analyze_permission_inconsistencies(scan_config)
        },
        'content_analysis': {
            'duplicate_files': detect_duplicate_files_by_hash(scan_config),
            'oversized_files': identify_unusually_large_files(scan_config),
            'obsolete_files': find_outdated_or_unused_files(scan_config),
            'format_inconsistencies': detect_files_with_wrong_extensions(scan_config),
            'naming_violations': find_naming_convention_violations(scan_config)
        },
        'performance_analysis': {
            'storage_utilization': calculate_storage_usage_patterns(scan_config),
            'access_patterns': analyze_file_access_frequency(scan_config),
            'growth_trends': track_directory_size_changes(scan_config),
            'fragmentation_analysis': assess_file_system_fragmentation(scan_config)
        }
    }
    
    # Generate anomaly severity scores
    anomaly_assessment = assess_anomaly_severity(scan_results)
    
    return scan_results, anomaly_assessment

Phase 2: Intelligent Cleanup Decision Making (1-3 minutes)

def generate_cleanup_recommendations(scan_results, anomaly_assessment):
    """
    Create prioritized cleanup plan with safety classifications
    """
    cleanup_plan = {
        'immediate_safe_actions': {
            'empty_directory_removal': {
                'directories': filter_safe_empty_directories(scan_results),
                'safety_level': 'safe',
                'automation': 'fully_automated',
                'estimated_impact': 'minimal_risk_high_benefit'
            },
            'broken_symlink_cleanup': {
                'symlinks': identify_broken_symlinks(scan_results),
                'safety_level': 'safe',
                'automation': 'fully_automated',
                'estimated_impact': 'no_risk_moderate_benefit'
            },
            'permission_standardization': {
                'files': find_permission_inconsistencies(scan_results),
                'safety_level': 'safe',
                'automation': 'fully_automated',
                'estimated_impact': 'low_risk_high_benefit'
            }
        },
        'scheduled_moderate_actions': {
            'duplicate_file_consolidation': {
                'duplicates': prioritize_duplicate_removal(scan_results),
                'safety_level': 'moderate',
                'automation': 'automated_with_confirmation',
                'estimated_impact': 'moderate_risk_high_benefit'
            },
            'obsolete_file_archival': {
                'files': identify_archival_candidates(scan_results),
                'safety_level': 'moderate',
                'automation': 'automated_with_notification',
                'estimated_impact': 'moderate_risk_moderate_benefit'
            },
            'large_file_compression': {
                'files': find_compression_candidates(scan_results),
                'safety_level': 'moderate',
                'automation': 'automated_with_backup',
                'estimated_impact': 'low_risk_high_benefit'
            }
        },
        'manual_review_actions': {
            'structural_reorganization': {
                'recommendations': suggest_directory_restructuring(scan_results),
                'safety_level': 'cautious',
                'automation': 'human_approval_required',
                'estimated_impact': 'high_risk_high_benefit'
            },
            'naming_convention_updates': {
                'violations': identify_naming_violations(scan_results),
                'safety_level': 'cautious',
                'automation': 'human_review_required',
                'estimated_impact': 'moderate_risk_moderate_benefit'
            }
        }
    }
    
    return cleanup_plan

Phase 3: Automated Cleanup Execution (5-30 minutes)

def execute_automated_cleanup(cleanup_plan):
    """
    Execute safe cleanup operations with comprehensive logging
    """
    execution_results = {
        'immediate_actions': {
            'executed_operations': [],
            'success_count': 0,
            'failure_count': 0,
            'space_recovered': 0,
            'files_processed': 0
        },
        'scheduled_actions': {
            'queued_operations': [],
            'confirmation_required': [],
            'estimated_completion': None
        },
        'manual_actions': {
            'flagged_for_review': [],
            'recommendations_generated': [],
            'priority_assignments': []
        }
    }
    
    # Execute immediate safe actions
    for action_type, action_config in cleanup_plan['immediate_safe_actions'].items():
        try:
            result = execute_cleanup_action(action_type, action_config)
            execution_results['immediate_actions']['executed_operations'].append(result)
            execution_results['immediate_actions']['success_count'] += 1
            execution_results['immediate_actions']['space_recovered'] += result.get('space_saved', 0)
            execution_results['immediate_actions']['files_processed'] += result.get('files_affected', 0)
        except Exception as e:
            log_cleanup_error(action_type, e)
            execution_results['immediate_actions']['failure_count'] += 1
    
    # Queue scheduled actions
    for action_type, action_config in cleanup_plan['scheduled_moderate_actions'].items():
        scheduled_operation = create_scheduled_operation(action_type, action_config)
        execution_results['scheduled_actions']['queued_operations'].append(scheduled_operation)
    
    # Flag manual review actions
    for action_type, action_config in cleanup_plan['manual_review_actions'].items():
        review_item = create_review_item(action_type, action_config)
        execution_results['manual_actions']['flagged_for_review'].append(review_item)
    
    return execution_results

Advanced Deduplication Engine

Intelligent Duplicate Detection

def advanced_duplicate_detection(file_list, detection_config):
    """
    Multi-level duplicate detection with intelligent consolidation
    """
    deduplication_analysis = {
        'exact_duplicates': {
            'detection_method': 'sha256_hash_comparison',
            'candidates': find_exact_hash_matches(file_list),
            'consolidation_strategy': 'keep_newest_or_best_location',
            'safety_level': 'high'
        },
        'near_duplicates': {
            'detection_method': 'fuzzy_hash_similarity',
            'similarity_threshold': 0.95,
            'candidates': find_similar_files(file_list, 0.95),
            'consolidation_strategy': 'human_review_required',
            'safety_level': 'moderate'
        },
        'content_similarity': {
            'detection_method': 'text_content_analysis',
            'similarity_threshold': 0.90,
            'candidates': find_content_similar_files(file_list),
            'consolidation_strategy': 'suggest_merge_or_archive',
            'safety_level': 'low'
        },
        'structural_similarity': {
            'detection_method': 'ast_comparison_for_code',
            'similarity_threshold': 0.85,
            'candidates': find_structurally_similar_code(file_list),
            'consolidation_strategy': 'refactoring_opportunity',
            'safety_level': 'manual_review'
        }
    }
    
    # Prioritize duplicates by impact and safety
    prioritized_duplicates = prioritize_duplicate_removal(deduplication_analysis)
    
    return prioritized_duplicates

Smart Consolidation Strategies

consolidation_strategies:
  keep_newest:
    criteria: "most_recent_modification_date"
    backup: "create_backup_of_older_versions"
    verification: "verify_content_integrity_after_removal"
    
  keep_best_location:
    criteria: "most_appropriate_directory_based_on_classification"
    relocation: "move_duplicates_to_optimal_location"
    linking: "create_hard_links_for_frequently_accessed_duplicates"
    
  keep_largest:
    criteria: "file_with_most_complete_content"
    analysis: "compare_file_sizes_and_content_completeness"
    validation: "ensure_no_data_loss_during_consolidation"
    
  create_hardlinks:
    criteria: "identical_files_in_different_logical_locations"
    implementation: "filesystem_level_hard_link_creation"
    monitoring: "track_hard_link_integrity_over_time"

Comprehensive Hygiene Reporting

Executive Hygiene Report Generation

def generate_comprehensive_hygiene_report(scan_results, cleanup_results):
    """
    Create detailed hygiene report with actionable insights
    """
    hygiene_report = {
        'executive_summary': {
            'overall_health_score': calculate_overall_health_score(scan_results),
            'critical_issues_resolved': count_critical_issues_resolved(cleanup_results),
            'storage_space_optimized': calculate_space_savings(cleanup_results),
            'performance_improvements': assess_performance_gains(cleanup_results),
            'trend_analysis': analyze_hygiene_trends_over_time(scan_results)
        },
        'detailed_findings': {
            'structural_health': {
                'directory_compliance_score': assess_structure_compliance(scan_results),
                'empty_directories_cleaned': count_empty_directories_removed(cleanup_results),
                'permission_issues_resolved': count_permission_fixes(cleanup_results),
                'broken_links_repaired': count_symlink_fixes(cleanup_results)
            },
            'content_optimization': {
                'duplicate_files_consolidated': count_duplicates_removed(cleanup_results),
                'obsolete_files_archived': count_files_archived(cleanup_results),
                'storage_efficiency_gained': calculate_storage_efficiency(cleanup_results),
                'naming_violations_flagged': count_naming_issues(scan_results)
            },
            'performance_metrics': {
                'scan_duration': measure_scan_performance(scan_results),
                'cleanup_efficiency': measure_cleanup_performance(cleanup_results),
                'system_impact': assess_system_resource_usage(cleanup_results),
                'user_disruption': measure_workflow_impact(cleanup_results)
            }
        },
        'actionable_recommendations': {
            'immediate_priorities': generate_immediate_action_items(scan_results),
            'scheduled_maintenance': create_maintenance_schedule(scan_results),
            'strategic_improvements': suggest_long_term_optimizations(scan_results),
            'automation_opportunities': identify_automation_potential(scan_results)
        },
        'trend_analysis': {
            'growth_patterns': analyze_directory_growth_trends(scan_results),
            'health_degradation': track_hygiene_score_changes(scan_results),
            'optimization_effectiveness': measure_cleanup_impact_over_time(cleanup_results),
            'predictive_insights': forecast_future_maintenance_needs(scan_results)
        }
    }
    
    return hygiene_report

Proactive Maintenance Scheduling

Intelligent Maintenance Orchestration

def optimize_maintenance_scheduling(project_context, historical_data):
    """
    Create intelligent maintenance schedule based on project patterns
    """
    maintenance_schedule = {
        'continuous_monitoring': {
            'frequency': 'real_time',
            'triggers': ['critical_permission_issues', 'broken_symlinks', 'security_violations'],
            'response_time': 'immediate',
            'automation_level': 'fully_automated'
        },
        'daily_maintenance': {
            'frequency': 'every_24_hours',
            'optimal_time': determine_low_activity_period(historical_data),
            'operations': ['temp_file_cleanup', 'log_rotation', 'cache_management'],
            'duration_estimate': '5-15_minutes'
        },
        'weekly_deep_clean': {
            'frequency': 'every_7_days',
            'optimal_day': determine_optimal_maintenance_day(project_context),
            'operations': ['duplicate_detection', 'obsolete_file_identification', 'structure_validation'],
            'duration_estimate': '30-60_minutes'
        },
        'monthly_optimization': {
            'frequency': 'every_30_days',
            'comprehensive_scope': True,
            'operations': ['trend_analysis', 'performance_optimization', 'strategic_recommendations'],
            'duration_estimate': '1-2_hours'
        }
    }
    
    return maintenance_schedule

Success Metrics

Hygiene Quality Standards

• ✅ Structure Compliance: 95%+ adherence to project template standards

• ✅ Duplicate Reduction: Maintain less than 2% duplicate file ratio

• ✅ Empty Directory Management: Zero empty directories older than grace period

• ✅ Permission Consistency: 100% consistent permissions within directory trees

Performance Standards

• ✅ Scan Efficiency: Complete full project scan within 10 minutes for typical projects

• ✅ Cleanup Speed: Process 1000+ files per minute during cleanup operations

• ✅ Resource Optimization: Maintain less than 10% CPU usage during routine operations

• ✅ Storage Savings: Achieve 15%+ storage space optimization through cleanup

User Experience Standards

• ✅ Zero Disruption: No interruption to active workflows during maintenance

• ✅ Report Clarity: 90%+ user satisfaction with report usefulness and actionability

• ✅ Automation Reliability: 99%+ success rate for automated cleanup operations

• ✅ Predictive Accuracy: 85%+ accuracy in predicting maintenance needs

Integration Points

Squad Coordination

squad_feedback_integration:
  structuro_insights:
    - template_effectiveness_analysis
    - directory_usage_pattern_feedback
    - structure_optimization_recommendations
    
  classifico_feedback:
    - classification_accuracy_validation
    - misplaced_file_pattern_analysis
    - rule_improvement_suggestions
    
  locomoto_coordination:
    - operation_success_rate_monitoring
    - performance_bottleneck_identification
    - system_optimization_opportunities

This task ensures continuous project health through intelligent monitoring, automated cleanup, and proactive optimization, maintaining pristine directory structures that support long-term project success and team productivity.