Merge pull request #5318 from wazuh/merge-4.9.0-into-master

Merge 4.9.0 into master
wazuh · Apr 30, 2024 · d6616bf · d6616bf
2 parents c497eba + e74423c
commit d6616bf
Show file tree

Hide file tree

Showing 28 changed files with 1,197 additions and 606 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -8,8 +8,10 @@ All notable changes to this project will be documented in this file.
 
 ### Added
 
-- Add Workflow module to Wazuh-qa repository ([#4990](https://github.com/wazuh/wazuh-qa/pull/4990)) \- (Tests)
 - Add integration tests for Update field to CPE_Helper. ([#4574](https://github.com/wazuh/wazuh-qa/pull/4574)) \- (Core)
+- Updated integration tests README ([#4742](https://github.com/wazuh/wazuh-qa/pull/4742)) \- (Framework)
+- Removed configobj library from requirements.txt ([#4803](https://github.com/wazuh/wazuh-qa/pull/4803)) \- (Framework)
+- Add Workflow module to Wazuh-qa repository ([#4990](https://github.com/wazuh/wazuh-qa/pull/4990)) \- (Tests)
 
 ### Changed
 
@@ -18,24 +20,16 @@ All notable changes to this project will be documented in this file.
 ### Fixed
 
 - Add an IT to check that the agent erases its wazuh-agent.state file ([#4716](https://github.com/wazuh/wazuh-qa/pull/4716)) \- (Core)
-
-## [4.8.2] - TBD
-
-## [4.8.1] - TBD
-
-### Changed
-
-- Removed configobj library from requirements.txt ([#4803](https://github.com/wazuh/wazuh-qa/pull/4803)) \- (Framework)
-- Updated integration tests README ([#4742](https://github.com/wazuh/wazuh-qa/pull/4742)) \- (Framework)
-
-### Fixed
-
 - Fix manager_agent system tests environment ([#4808](https://github.com/wazuh/wazuh-qa/pull/4808)) \- (Framework)
+- Fixed agent_simulator response for active-response configuration commands. ([#4895](https://github.com/wazuh/wazuh-qa/pull/4895)) \- (Framework)
 
 ## [4.8.0] - TBD
 
 ### Added
 
+- Add functionality to obtain statistics and metrics from the indexer. ([#5090](https://github.com/wazuh/wazuh-qa/pull/5090)) \- (Framework)
+- Add support for the installation/uninstallation of npm packages ([#5092](https://github.com/wazuh/wazuh-qa/pull/5092)) \- (Tests)
+- Add alert.json file to Vulnerability Detector E2E test report ([#5147](https://github.com/wazuh/wazuh-qa/pull/5147)) \- (Framework)
 - Add documentation about markers for system tests ([#5080](https://github.com/wazuh/wazuh-qa/pull/5080)) \- (Documentation)
 - Add AWS Custom Buckets Integration tests ([#4675](https://github.com/wazuh/wazuh-qa/pull/4675)) \- (Framework + Tests)
 - Add Vulnerability Detector end to end tests ([#4878](https://github.com/wazuh/wazuh-qa/pull/4878)) \- (Framework + Tests)
@@ -52,6 +46,8 @@ All notable changes to this project will be documented in this file.
 
 ### Changed
 
+- Replace timestamp filter with vulnerabilities detected_at field.([#5266](https://github.com/wazuh/wazuh-qa/pull/5266)) \- (Framework + Tests)
+- Changes macOS packages with new ones that generate vulnerabilities ([#5174](https://github.com/wazuh/wazuh-qa/pull/5174)) \- (Tests)
 - Refactor initial scan Vulnerability E2E tests ([#5081](https://github.com/wazuh/wazuh-qa/pull/5081)) \- (Framework + Tests)
 - Update Packages in TestScanSyscollectorCases ([#4997](https://github.com/wazuh/wazuh-qa/pull/4997)) \- (Framework + Tests)
 - Reduced test_shutdown_message runtime ([#4986](https://github.com/wazuh/wazuh-qa/pull/4986)) \- (Tests)
@@ -79,6 +75,15 @@ All notable changes to this project will be documented in this file.
 
 ### Fixed
 
+- Fix packages in Windows and macOS upgrade cases ([#5223](https://github.com/wazuh/wazuh-qa/pull/5223)) \- (Framework + Tests)
+- Fix vulnerabilities and add new packages to Vulnerability Detector E2E tests ([#5234](https://github.com/wazuh/wazuh-qa/pull/5234)) \- (Tests)
+- Fix provision macOS endpoints with npm ([#5128](https://github.com/wazuh/wazuh-qa/pull/5158)) \- (Tests)
+- Fix timestamps alerts and logs filter ([#5157](https://github.com/wazuh/wazuh-qa/pull/5157)) \- (Framework + Tests)
+- Fix macOS and Windows agents timezone ([#5178](https://github.com/wazuh/wazuh-qa/pull/5178)) \- (Framework)
+- Fix Vulnerability Detector E2E tests by adding description to all tests ([#5151](https://github.com/wazuh/wazuh-qa/pull/5151)) \- (Tests)
+- Fix parser for non package vulnerabilities ([#5146](https://github.com/wazuh/wazuh-qa/pull/5146)) \- (Framework)
+- Fix remote_operations_handler functions to Vulnerability Detector E2E tests ([#5155](https://github.com/wazuh/wazuh-qa/pull/5155)) \- (Framework)
+- Fix enrollment cluster system tests ([#5134](https://github.com/wazuh/wazuh-qa/pull/5134)) \- (Tests)
 - Fix `test_synchronization` system test ([#5089](https://github.com/wazuh/wazuh-qa/pull/5089)) \- (Framework + Tests)
 - Fix number of files and their size for `test_zip_size_limit` ([#5133](https://github.com/wazuh/wazuh-qa/pull/5133)) \- (Tests)
 - Fix test_shutdown_message system test ([#5087](https://github.com/wazuh/wazuh-qa/pull/5087)) \- (Tests)
@@ -106,6 +111,11 @@ All notable changes to this project will be documented in this file.
 - Fix test cluster performance. ([#4780](https://github.com/wazuh/wazuh-qa/pull/4780)) \- (Framework)
 - Fixed the graphic generation for the logcollectord statistics files. ([#5021](https://github.com/wazuh/wazuh-qa/pull/5021)) \- (Framework)
 
+
+## [4.7.4] - 29/04/2024
+
+- No changes
+
 ## [4.7.3] - 04/03/2024
 
 ### Changed
@@ -716,4 +726,4 @@ Release report: https://github.com/wazuh/wazuh/issues/13321
 - Avoid problematic race-condition on VD integration tests for Windows [#1047](https://github.com/wazuh/wazuh-qa/pull/1047)
 - QA Integration tests stabilization [#1002](https://github.com/wazuh/wazuh-qa/pull/1002)
 ### Deleted
-- Deleted `behind_proxy_server` API config test. ([#1065](https://github.com/wazuh/wazuh-qa/pull/1065))
+- Deleted `behind_proxy_server` API config test. ([#1065](https://github.com/wazuh/wazuh-qa/pull/1065))
diff --git a/deps/wazuh_testing/wazuh_testing/end_to_end/indexer_api.py b/deps/wazuh_testing/wazuh_testing/end_to_end/indexer_api.py
@@ -18,11 +18,78 @@
 from wazuh_testing.tools.system import HostManager
 
 
-STATE_INDEX_NAME = 'wazuh-vulnerabilities-states'
+STATE_INDEX_NAME = 'wazuh-states-vulnerabilities'
+
+
+def create_vulnerability_states_indexer_filter(target_agent: str | None = None,
+                                               greater_than_timestamp: str | None = None) -> dict:
+    """Create a filter for the Indexer API for the vulnerability state index.
+
+    Args:
+        target_agent: The target agent to filter on.
+        greater_than_timestamp: The timestamp to filter on.
+
+    Returns:
+        dict: A dictionary containing the filter.
+    """
+    timestamp_filter = None
+    if greater_than_timestamp:
+        timestamp_filter = {
+                'greater_than_timestamp': greater_than_timestamp,
+                'timestamp_name': 'vulnerability.detected_at'
+        }
+
+    return _create_filter(target_agent, timestamp_filter)
+
+
+def create_alerts_filter(target_agent: str | None = None, greater_than_timestamp: str | None = None) -> dict:
+    """Create a filter for the Indexer API for the alerts index.
+
+    Args:
+        target_agent: The target agent to filter on.
+        greater_than_timestamp: The timestamp to filter on.
+
+    Returns:
+        dict: A dictionary containing the filter.
+    """
+    timestamp_filter = None
+    if greater_than_timestamp:
+        timestamp_filter = {
+                'greater_than_timestamp': greater_than_timestamp,
+                'timestamp_name': '@timestamp'
+        }
+
+    return _create_filter(target_agent, timestamp_filter)
+
+
+def _create_filter(target_agent: str | None = None, timestamp_filter: dict | None = None) -> dict:
+    """Create a filter for the Indexer API.
+
+    Args:
+        target_agent: The target agent to filter on.
+        greater_than_timestamp: The timestamp to filter on.
+        timestamp_field: The timestamp field to filter on.
+
+    Returns:
+        dict: A dictionary containing the filter.
+    """
+    filter = {
+        'bool': {
+            'must': []
+        }
+    }
+    if timestamp_filter:
+        timestamp_field = timestamp_filter['timestamp_name']
+        greater_than_timestamp = timestamp_filter['greater_than_timestamp']
+        filter['bool']['must'].append({'range': {timestamp_field: {'gte': greater_than_timestamp}}})
+    if target_agent:
+        filter['bool']['must'].append({'match': {'agent.name': target_agent}})
+
+    return filter
 
 
 def get_indexer_values(host_manager: HostManager, credentials: dict = {'user': 'admin', 'password': 'changeme'},
-                       index: str = 'wazuh-alerts*', greater_than_timestamp=None, agent: str = '') -> Dict:
+                       index: str = 'wazuh-alerts*', filter: dict | None = None, size: int = 10000) -> Dict:
     """
     Get values from the Wazuh Indexer API.
 
@@ -31,65 +98,48 @@ def get_indexer_values(host_manager: HostManager, credentials: dict = {'user': '
         credentials (Optional): A dictionary containing the Indexer credentials. Defaults to
                                  {'user': 'admin', 'password': 'changeme'}.
         index (Optional): The Indexer index name. Defaults to 'wazuh-alerts*'.
-        greater_than_timestamp (Optional): The timestamp to filter the results. Defaults to None.
-        agent (Optional): The agent name to filter the results. Defaults to ''.
+        filter (Optional): A dictionary containing the query filter. Defaults to None.
+        size (Optional): The number of results to retrieve. Defaults to 10000.
 
     Returns:
        Dict: A dictionary containing the values retrieved from the Indexer API.
     """
     logging.info(f"Getting values from the Indexer API for index {index}")
 
     url = f"https://{host_manager.get_master_ip()}:9200/{index}/_search"
-    headers = {
-        'Content-Type': 'application/json',
-    }
-
-    data = {
-        "query": {
-            "match_all": {}
-        }
-    }
-
-    if greater_than_timestamp and agent:
-        query = {
-                "bool": {
-                    "must": [
-                        {"range": {"@timestamp": {"gte": f"{greater_than_timestamp}"}}},
-                        {"match": {"agent.name": f"{agent}"}}
-                    ]
-                }
-        }
-
-        data['query'] = query
-    elif greater_than_timestamp:
-        query = {
-                "bool": {
-                    "must": [
-                        {"range": {"@timestamp": {"gte": f"{greater_than_timestamp}"}}}
-                    ]
-                }
-        }
 
-        data['query'] = query
-    elif agent:
-        query = {
-                "bool": {
-                    "must": [
-                        {"match": {"agent.name": f"{agent}"}}
-                    ]
-                }
-        }
+    data = {}
+    param = {'size': size}
+    headers = {'Content-Type': 'application/json'}
 
-        data['query'] = query
-
-    param = {
-        'pretty': 'true',
-        'size': 10000,
-    }
+    if filter:
+        data['query'] = filter
 
     response = requests.get(url=url, params=param, verify=False,
                             auth=requests.auth.HTTPBasicAuth(credentials['user'], credentials['password']),
                             headers=headers,
                             json=data)
 
     return response.json()
+
+
+def delete_index(host_manager: HostManager, credentials: dict = {'user': 'admin', 'password': 'changeme'},
+                 index: str = 'wazuh-alerts*'):
+    """
+    Delete index from the Wazuh Indexer API.
+
+    Args:
+        host_manager: An instance of the HostManager class containing information about hosts.
+        credentials (Optional): A dictionary containing the Indexer credentials. Defaults to
+                                 {'user': 'admin', 'password': 'changeme'}.
+        index (Optional): The Indexer index name. Defaults to 'wazuh-alerts*'.
+    """
+    logging.info(f"Deleting {index} index")
+
+    url = f"https://{host_manager.get_master_ip()}:9200/{index}/"
+    headers = {
+        'Content-Type': 'application/json',
+    }
+
+    requests.delete(url=url, verify=False,
+                    auth=requests.auth.HTTPBasicAuth(credentials['user'], credentials['password']), headers=headers)
diff --git a/deps/wazuh_testing/wazuh_testing/end_to_end/logs.py b/deps/wazuh_testing/wazuh_testing/end_to_end/logs.py
@@ -52,10 +52,29 @@ def get_hosts_logs(host_manager: HostManager, host_group: str = 'all') -> Dict[s
     - host_manager (HostManager): An instance of the HostManager class for managing remote hosts.
     - host_group (str, optional): The name of the host group where the files will be truncated.
       Default is 'all'.
+
+    Returns:
+    - host_logs (Dict[str, str]): Dictionary containing the logs from the ossec.log file of each host
     """
     host_logs = {}
     for host in host_manager.get_group_hosts(host_group):
         host_os_name = host_manager.get_host_variables(host)['os_name']
         host_logs[host] = host_manager.get_file_content(host, logs_filepath_os[host_os_name])
 
     return host_logs
+
+def get_hosts_alerts(host_manager: HostManager) -> Dict[str, str]:
+    """
+    Get the alerts in the alert.json file from the specified host group.
+
+    Parameters:
+    - host_manager (HostManager): An instance of the HostManager class for managing remote hosts.
+
+    Returns:
+    - host_alerts (Dict[str, str]): Dictionary containing the alerts from the alert.json file of each manager
+    """
+    host_alerts = {}
+    for host in host_manager.get_group_hosts("manager"):
+        host_alerts[host] = host_manager.get_file_content(host, ALERTS_JSON_PATH)
+
+    return host_alerts
diff --git a/deps/wazuh_testing/wazuh_testing/end_to_end/regex.py b/deps/wazuh_testing/wazuh_testing/end_to_end/regex.py
@@ -46,10 +46,10 @@
         'parameters': ['HOST_NAME', 'CVE', 'PACKAGE_NAME', 'PACKAGE_VERSION', 'ARCHITECTURE']
     },
     'vuln_affected': {
-        'regex':  'CVE.*? affects.*"?'
+        'regex':  'CVE.* affects.*"?'
     },
     'vuln_mitigated': {
-        'regex': "The .* that affected .* was solved due to a package removal"
+        'regex': "The .* that affected .* was solved due to a package removal.*"
     }
 }