Merge pull request #9 from LoH-lu/beta

Beta merge with main

README.md

@@ -1,31 +1,69 @@
 # netbox-nmap-scan
 
-This is a simple Python scripts that achieve the purpose of keeping an updated list of IP Address which are active/responding in your network.
-To achieve that we are using nmap.
+Automatically maintain an up-to-date inventory of active IP addresses in your network using Netbox and nmap. This Python-based tool scans your network prefixes and keeps your Netbox instance synchronized with the current state of your network.
 
-By default the scripts is scanning all prefixes with the active status inside your Netbox instance.
-If you don't want a prefix to get scan, create a new tag 'Disable Automatic Scanning'
+## Features
 
-![image](https://github.com/henrionlo/netbox-nmap-scan/assets/139378145/b7a223ae-3a55-42cb-8f28-87d282e103c8)
+- Automatic scanning of all active prefixes in Netbox
+- Custom tag support for excluding prefixes from scanning
+- Tracking of last scan time for each IP address
+- DNS resolution support
+- Tested with Python 3.12.6 - 3.13.1 and Netbox 4.1.10
 
-Create the tag 'autoscan', this will allow you to quickly know which IP Addresses has been added by the script.
+## Prerequisites
 
-![image](https://github.com/henrionlo/netbox-nmap-scan/assets/139378145/435cec58-1f92-42f2-b4eb-1448a4d22161)
+- Python 3.12.6 or later
+- Netbox 4.1.10 or later
+- nmap installed on your system
+- Required Python packages (listed in requirements.txt)
 
-And create the following custom field in Customization, this way you can see when was the last time an ip address has been pinged by the scanning engine.
+## Setup
 
-![image](https://github.com/LoH-lu/netbox-nmap-scan/assets/139378145/c812ee55-71d0-4d8e-9b14-f337a5d867a5)
+1. Create the following Netbox configurations:
 
-The more prefixes you want to scan, the more time it will require to finish.
+   ### Tags
+   - `autoscan`: Identifies IP addresses added by this script
+   - `Disable Automatic Scanning`: Add this tag to prefixes you want to exclude from scanning
+
+   ![Disable Scanning Tag Configuration](https://github.com/henrionlo/netbox-nmap-scan/assets/139378145/b7a223ae-3a55-42cb-8f28-87d282e103c8)
+
+   ![Autoscan Tag Configuration](https://github.com/henrionlo/netbox-nmap-scan/assets/139378145/435cec58-1f92-42f2-b4eb-1448a4d22161)
 
-Tested and working with Python 3.12.2 - 3.12.4 and Netbox 3.6.x - 4.0.x
+   ### Custom Fields
+   Add a custom field to track the last scan time for each IP address:
+
+   ![Last Scan Time Custom Field](https://github.com/LoH-lu/netbox-nmap-scan/assets/139378145/c812ee55-71d0-4d8e-9b14-f337a5d867a5)
 
-The How-To are located in https://github.com/henrionlo/netbox-nmap-scan/wiki
+2. Follow the detailed installation guide in our [Wiki](https://github.com/henrionlo/netbox-nmap-scan/wiki)
 
-TODO
-- Add DNS server selection for the nmap command in the ini file (if required to have a different one from the system DNS running the script)
-- Allow users to disable the DNS part of the script and only run the regular nmap command
-- Cleanup of code and import
-- Adding more description
-- Better logging of errors and debug output
-- All-in-One script for easier setup
+## Usage
+
+The script will scan all prefixes with active status in your Netbox instance by default. Scanning time increases with the number of prefixes being scanned.
+
+For detailed usage instructions and examples, please refer to our [Wiki](https://github.com/henrionlo/netbox-nmap-scan/wiki).
+
+## Performance Considerations
+
+- Scanning time scales with the number of prefixes
+- Consider scheduling scans during off-peak hours for large networks
+- Use the `Disable Automatic Scanning` tag strategically to optimize scan times
+
+## Roadmap
+
+- [ ] DNS server configuration in INI file for custom DNS resolution
+- [X] Option to disable DNS resolution functionality
+- [X] Toggle for last scan time tracking
+- [X] Toggle for the progress bar display while importing
+- [ ] Toggle for the logger in the Python console
+- [X] All-in-One setup script for easier deployment
+
+## Contributing
+
+Contributions are welcome! Please feel free to submit a Pull Request.
+
+## Support
+
+For issues, questions, or contributions, please:
+1. Check our [Wiki](https://github.com/henrionlo/netbox-nmap-scan/wiki)
+2. Open an issue in this repository
+3. Submit a pull request with your proposed changes

main.py

@@ -0,0 +1,123 @@
+#!/usr/bin/env python3
+"""
+Script runner that executes multiple Python scripts in sequence.
+This module provides functionality to run multiple Python scripts in order,
+with comprehensive logging of execution results.
+"""
+
+import subprocess
+import sys
+import os
+import logging
+from datetime import datetime
+from typing import List
+
+# Define the directory for logs and scripts
+SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
+LOG_DIR = os.path.join(SCRIPT_DIR, 'logs')
+
+# Ensure the log directory exists
+os.makedirs(LOG_DIR, exist_ok=True)
+
+def setup_logging() -> logging.Logger:
+    """
+    Configure logging with both file and console handlers.
+
+    Returns:
+        logging.Logger: Configured logger instance
+    """
+    # Create logger
+    logger = logging.getLogger(__name__)
+    logger.setLevel(logging.DEBUG)
+
+    # Create formatters
+    file_formatter = logging.Formatter(
+        '%(asctime)s - %(levelname)s - [%(filename)s:%(lineno)d] - %(message)s'
+    )
+    console_formatter = logging.Formatter(
+        '%(asctime)s - %(levelname)s - %(message)s'
+    )
+
+    # Create file handlers
+    timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
+    debug_handler = logging.FileHandler(
+        os.path.join(LOG_DIR, f'script_execution_debug_{timestamp}.log')
+    )
+    debug_handler.setLevel(logging.DEBUG)
+    debug_handler.setFormatter(file_formatter)
+
+    error_handler = logging.FileHandler(
+        os.path.join(LOG_DIR, f'script_execution_error_{timestamp}.log')
+    )
+    error_handler.setLevel(logging.ERROR)
+    error_handler.setFormatter(file_formatter)
+
+    # Create console handler
+    console_handler = logging.StreamHandler()
+    console_handler.setLevel(logging.INFO)
+    console_handler.setFormatter(console_formatter)
+
+    # Add handlers to logger
+    logger.addHandler(debug_handler)
+    logger.addHandler(error_handler)
+    logger.addHandler(console_handler)
+
+    return logger
+
+def run_script(script_name: str, logger: logging.Logger) -> bool:
+    """
+    Executes a Python script using the current Python interpreter.
+
+    Args:
+        script_name (str): The name of the script to run.
+        logger (logging.Logger): The logger instance for logging messages.
+
+    Returns:
+        bool: True if the script runs successfully, False otherwise.
+    """
+    logger.info(f"Running {script_name}...")
+
+    try:
+        # Run the script using the current Python interpreter
+        result = subprocess.run(
+            [sys.executable, script_name],
+            capture_output=True,
+            text=True,
+            check=True
+        )
+        logger.info(f"{script_name} completed successfully.")
+        logger.debug(f"Output:\n{result.stdout}")
+        return True
+    except subprocess.CalledProcessError as e:
+        logger.error(f"Error running {script_name}:")
+        logger.error(e.stderr)  # Log the error message if the script fails
+        return False
+
+def main():
+    """
+    Main function to run a list of Python scripts sequentially.
+
+    The function will stop execution if any script fails, preventing subsequent
+    scripts from running if an error is encountered.
+    """
+    logger = setup_logging()
+
+    # List of scripts to execute in order
+    scripts: List[str] = [
+        "netbox_export.py",
+        "network_scan.py",
+        "scan_processor.py",
+        "netbox_import.py"
+    ]
+
+    # Iterate over the list of scripts and run each one
+    for script in scripts:
+        if not run_script(script, logger):
+            logger.error(f"Execution stopped due to an error in {script}")
+            break  # Stop execution if a script fails
+    else:
+        logger.info("All scripts executed successfully.")
+
+if __name__ == "__main__":
+    # Run the main function if the script is executed directly
+    main()