For the task of anomaly detection, there is a problem with the “detection adjustment” method.

[caocaort]

Hello Author,Regarding the anomaly_detection task, there is code related to detection adjustment in your code:

def adjustment(gt, pred):
    anomaly_state = False  # Initialize anomaly state as False
    # Loop through all elements in the ground truth (gt)
    for i in range(len(gt)):
        # Check if there's a true positive anomaly detected and not currently in an anomaly state
        if gt[i] == 1 and pred[i] == 1 and not anomaly_state:
            anomaly_state = True  # Set the anomaly state to True
            # Look backwards to adjust predictions preceding the detected anomaly
            for j in range(i, 0, -1):  # Iterate backwards from the current index
                if gt[j] == 0:  # Stop if a non-anomaly is found in ground truth
                    break
                else:
                    # Set prediction to anomaly if it was originally missed
                    if pred[j] == 0:
                        pred[j] = 1
            # Look forwards to adjust predictions following the detected anomaly
            for j in range(i, len(gt)):
                if gt[j] == 0:  # Stop if a non-anomaly is found in ground truth
                    break
                else:
                    # Set prediction to anomaly if it was originally missed
                    if pred[j] == 0:
                        pred[j] = 1
        # Reset anomaly state to False if current ground truth is not an anomaly
        elif gt[i] == 0:
            anomaly_state = False

        # If in anomaly state, ensure prediction is marked as anomaly
        if anomaly_state:
            pred[i] = 1
    
    # Return adjusted ground truth and predictions
    return gt, pred

However, in the industrial data, there are no labels. The detection adjustment method is based on labels to further filter. I think there is a problem with this evaluation method.

And after I removed the part of the code related to detection adjustment, the performance of the model dropped sharply.

Therefore, how can effective anomaly detection be carried out for time series data without labels?

[caocaort]

I believe the method mentioned in the references should not be used here:

“”“
In real applications, the human operators generally do not care about the point-wise metrics. It is acceptable for an algorithm to trigger an alert for any point in a contiguous anomaly segment, if the delay is not too long. Some metrics for anomaly detection have been proposed to accommodate this preference, e.g., [22], but most are not widely accepted, likely because they are too complicated. We instead use a simple strategy: if any point in an anomaly segment in the ground truth can be detected by a chosen threshold, we say this segment is detected correctly, and all points in this segment are treated as if they can be detected by this threshold. Meanwhile, the points outside the anomaly segments are treated as usual. The precision, recall, AUC, F-score and best F-score are then computed accordingly. This approach is illustrated in Fig 7。
“””----https://arxiv.org/pdf/1802.03903.pdf

[caocaort]

It should be noted that this method is currently highly controversial:
DAMO-DI-ML/KDD2023-DCdetector#35
thuml/Anomaly-Transformer#65

Kim, S.、Choi, K.、Choi, HS、Lee, B. 和 Yoon, S.（2022 年 6 月）。对时间序列异常检测进行严格评估。AAAI 人工智能会议记录（第 36 卷，第 7 期，第 7194-7201 页）。可以在这里找到。

[wuhaixu2016]

Many thanks for your valuable comments. Yes, anomaly_adjustment will make the results look better. We think it can be used as a metric for series-wise anomaly detection. We have a discussion here: thuml/Anomaly-Transformer#14