Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NET-938 Modify S2629: Add benchmarks #4602

Merged
merged 2 commits into from
Jan 8, 2025
Merged

NET-938 Modify S2629: Add benchmarks #4602

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dbaa679
NET-938 Modify S2629: Add benchmarks
sebastien-marichal Jan 8, 2025
dccd3b8
Remov median column
sebastien-marichal Jan 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 178 additions & 1 deletion rules/S2629/csharp/rule.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,181 @@ logger.DebugFormat("The value of the parameter is: {Parameter}.", parameter);

=== Documentation

* Microsoft Learn - https://learn.microsoft.com/en-us/dotnet/api/system.runtime.compilerservices.interpolatedstringhandlerattribute[InterpolatedStringHandlerArgumentAttribute]
* Microsoft Learn - https://learn.microsoft.com/en-us/dotnet/api/system.runtime.compilerservices.interpolatedstringhandlerattribute[InterpolatedStringHandlerArgumentAttribute]

=== Benchmarks

[options="header"]
|===
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the Median column needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not

| Method | Runtime | Mean | Standard Deviation | Median | Allocated
| | | | | |
| CastleCoreLoggingTemplateNotConstant | .NET 9.0 | 230.306 us | 2.7116 us | 230.669 us | 479200 B
| CastleCoreLoggingTemplateConstant | .NET 9.0 | 46.827 us | 1.4743 us | 46.741 us | 560000 B
| CastleCoreLoggingTemplateNotConstant | .NET Framework 4.8.1 | 1,060.413 us | 32.3559 us | 1,056.943 us | 1115276 B
| CastleCoreLoggingTemplateConstant | .NET Framework 4.8.1 | 93.697 us | 1.8201 us | 93.828 us | 561650 B
| | | | | |
| MSLoggingTemplateNotConstant | .NET 9.0 | 333.246 us | 12.9214 us | 331.407 us | 479200 B
| MSLoggingTemplateConstant | .NET 9.0 | 441.118 us | 68.7999 us | 456.558 us | 560000 B
| MSLoggingTemplateNotConstant | .NET Framework 4.8.1 | 1,542.076 us | 99.3423 us | 1,511.301 us | 1115276 B
| MSLoggingTemplateConstant | .NET Framework 4.8.1 | 698.071 us | 18.6319 us | 700.420 us | 561653 B
| | | | | |
| NLogLoggingTemplateNotConstant | .NET 9.0 | 178.789 us | 9.2528 us | 178.003 us | 479200 B
| NLogLoggingTemplateConstant | .NET 9.0 | 6.009 us | 1.3303 us | 6.112 us | -
| NLogLoggingTemplateNotConstant | .NET Framework 4.8.1 | 1,086.260 us | 44.1670 us | 1,082.003 us | 1115276 B
| NLogLoggingTemplateConstant | .NET Framework 4.8.1 | 25.132 us | 0.5666 us | 25.067 us | -
| | | | | |
| SerilogLoggingTemplateNotConstant | .NET 9.0 | 234.460 us | 7.4831 us | 232.205 us | 479200 B
| SerilogLoggingTemplateConstant | .NET 9.0 | 49.854 us | 1.8232 us | 49.799 us | -
| SerilogLoggingTemplateNotConstant | .NET Framework 4.8.1 | 1,103.939 us | 47.0203 us | 1,091.234 us | 1115276 B
| SerilogLoggingTemplateConstant | .NET Framework 4.8.1 | 35.752 us | 0.6022 us | 35.736 us | -
| | | | | |
| Log4NetLoggingTemplateNotConstant | .NET 9.0 | 255.754 us | 5.6046 us | 255.767 us | 479200 B
| Log4NetLoggingTemplateConstant | .NET 9.0 | 46.425 us | 1.7087 us | 46.535 us | 240000 B
| Log4NetLoggingTemplateNotConstant | .NET Framework 4.8.1 | 1,109.874 us | 23.4388 us | 1,107.015 us | 1115276 B
| Log4NetLoggingTemplateConstant | .NET Framework 4.8.1 | 92.305 us | 2.4161 us | 92.467 us | 240707 B
|===



==== Glossary

* https://en.wikipedia.org/wiki/Arithmetic_mean[Mean]
* https://en.wikipedia.org/wiki/Standard_deviation[Standard Deviation]

The results were generated by running the following snippet with https://github.com/dotnet/BenchmarkDotNet[BenchmarkDotNet]:

[source,csharp]
----
using Microsoft.Extensions.Logging;
using ILogger = Microsoft.Extensions.Logging.ILogger;

[Params(10_000)]
public int Iterations;

private ILogger ms_logger;
private Castle.Core.Logging.ILogger cc_logger;
private log4net.ILog l4n_logger;
private Serilog.ILogger se_logger;
private NLog.ILogger nl_logger;

[GlobalSetup]
public void GlobalSetup()
{
ms_logger = new LoggerFactory().CreateLogger<LoggingTemplates>();
cc_logger = new Castle.Core.Logging.NullLogFactory().Create("Castle.Core.Logging");
l4n_logger = log4net.LogManager.GetLogger(typeof(LoggingTemplates));
se_logger = Serilog.Log.Logger;
nl_logger = NLog.LogManager.GetLogger("NLog");
}

[BenchmarkCategory("Microsoft.Extensions.Logging")]
[Benchmark]
public void MSLoggingTemplateNotConstant()
{
for (int i = 0; i < Iterations; i++)
{
ms_logger.LogInformation($"Param: {i}");
}
}

[BenchmarkCategory("Microsoft.Extensions.Logging")]
[Benchmark]
public void MSLoggingTemplateConstant()
{
for (int i = 0; i < Iterations; i++)
{
ms_logger.LogInformation("Param: {Parameter}", i);
}
}

[BenchmarkCategory("Castle.Core.Logging")]
[Benchmark]
public void CastleCoreLoggingTemplateNotConstant()
{
for (int i = 0; i < Iterations; i++)
{
cc_logger.Info($"Param: {i}");
}
}

[BenchmarkCategory("Castle.Core.Logging")]
[Benchmark]
public void CastleCoreLoggingTemplateConstant()
{
for (int i = 0; i < Iterations; i++)
{
cc_logger.InfoFormat("Param: {Parameter}", i);
}
}

[BenchmarkCategory("log4net")]
[Benchmark]
public void Log4NetLoggingTemplateNotConstant()
{
for (int i = 0; i < Iterations; i++)
{
l4n_logger.Info($"Param: {i}");
}
}

[BenchmarkCategory("log4net")]
[Benchmark]
public void Log4NetLoggingTemplateConstant()
{
for (int i = 0; i < Iterations; i++)
{
l4n_logger.InfoFormat("Param: {Parameter}", i);
}
}

[BenchmarkCategory("Serilog")]
[Benchmark]
public void SerilogLoggingTemplateNotConstant()
{
for (int i = 0; i < Iterations; i++)
{
se_logger.Information($"Param: {i}");
}
}

[BenchmarkCategory("Serilog")]
[Benchmark]
public void SerilogLoggingTemplateConstant()
{
for (int i = 0; i < Iterations; i++)
{
se_logger.Information("Param: {Parameter}", i);
}
}

[BenchmarkCategory("NLog")]
[Benchmark]
public void NLogLoggingTemplateNotConstant()
{
for (int i = 0; i < Iterations; i++)
{
nl_logger.Info($"Param: {i}");
}
}

[BenchmarkCategory("NLog")]
[Benchmark]
public void NLogLoggingTemplateConstant()
{
for (int i = 0; i < Iterations; i++)
{
nl_logger.Info("Param: {Parameter}", i);
}
}
----

Hardware Configuration:

[source]
----
BenchmarkDotNet v0.14.0, Windows 10 (10.0.19045.5247/22H2/2022Update)
12th Gen Intel Core i7-12800H, 1 CPU, 20 logical and 14 physical cores
[Host] : .NET Framework 4.8.1 (4.8.9282.0), X64 RyuJIT VectorSize=256
.NET 9.0 : .NET 9.0.0 (9.0.24.52809), X64 RyuJIT AVX2
.NET Framework 4.8.1 : .NET Framework 4.8.1 (4.8.9282.0), X64 RyuJIT VectorSize=256
----
Loading