static string size bug in store_timescale #1395
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
It appears that logical types must be declared as a nested type in avro.
Avro metric names cannot include characters other than [\\.\\-A-Za-z0-9]. This change adds a service to provide this mapping to Avro storage plugins.
The LDMS metric values in rows presented to the commit function may contain metric names that are invalid. This change employs the Avro name mapping service to convert LDMS names to valid Avro names.
Any LDMS type that doesn't have an Avro equivalent is encoded in an Avro type with a logicalType annotation that indicates the type from which the value came.
Threads were unconditionally being created even when there were idle zap_io_threads available.
Update the store_avro_kakfa man page with the new Avro logicalType values
Resolve issue #1183 where multiple commas would be inserted into the json response object if a producer is disconnected
set_sec_mod regex=<regex> [uid=<uid>] [gid=<gid>] [perm=<perm>] set_sec_mod is an LDMSD configuration command to change UID, GID, and permission bits of LDMS sets using regular expressions. The command does not affect remote sets, sets aggregatred from a remote LDMSD. The new UID/GID/perm values only affect new clients and connections. That is, the aggregators that already have access to the sets will not lose access.
The fix makes the handler to handle the decomposition attribute.
The test sampler was incorrectly handling the component_id string, assuming that it would not have trailing whitespace or other non-numeric character.
[FIX] change schema name to aggregate data
ldmsd_plugins_usage checking for input 'all' must be done only if the input is not already null.
missing $ caused explicit --disable-Feature to be ignored in several cases. backport from OVIS-4, 876d87a
libcxi.h fails to include cassini_cntr_desc.h, which is almost certainly a bug in their header include tree. We include it explicitly until they fix things.
While not always calculable, the duration of tracked pids is of great interest and rather hard to compute downstream. This adds the duration field to process end messages and a format option to suppress it for sites not ready to migrate to the expanded schema. Unlike ldmsd, the status of ldms-notify cannot be determined remotely, so this also adds a heartbeat message, which by default is not generated.
Purge-track-dir cleans up files in the track-dir location which no longer match existing processes. The env vars for heartbeat, component_id and producer used to be ignored and are now followed. The option_dump routine now properly prints the long options and there influences.
…ler.c Extraneous %s in format strings cause wild pointer reads and possible crashes.
Also added support for using main and v* style tags in the future.
In ROCM 5.5.0, rdc.h has been fixed so that it compiles under gcc. As a result we can drop our hack to fix rdc.h on the fly during the build. The ROCM source tree has seen movement towards a more standardized structure, so also convert to using the AC_LIB_HAVE_LINKFLAGS macro. If ROCM is not installed in the standard search path, the path to librdc_bootstrap can be specified with a single --with-librdc_boostrap-prefix option. For instance: ./configure --enable-rdc --with-librdc_bootstrap-prefix=/opt/rocm-5.5.0
The following plugins were added 1) zfs_zpool Stastistics on zfs pools. 2) zfs_leafvdevs Statistic on virtual devices leaves only. 3) zfs_topvdevs Statidstics on top virtual devices in the device tree (parent only).
Refactor the code a bit more to allow a missing call to avro_value_iface_decref(). This should plug a memory leak.
In order to support avro schema evolution, add naive default values to each field (except the timestamp, which ldms metric sets will always contain). Technically for the default form of schema evolution in kafka, we only need to have defaults for any _new_ fields that we add when evolving a schema. However, we are not tracking the various iterations of the schema over time, so just setting defaults for all fields allows us greater flexibility.
The name alone of the avro schema in the kafka registry is insufficient to match an ldms schema. A schema in the kafka registry is uniquely identified by a combination of a name and an integer ID. The schema with the same name can evolve over time, and each iteration if given new unique ID. We want to _always_ generate an avro schema, and attempt to push it to the registry. When the schema hasn't changed, the kafka registry handles the attempt to add a duplicate gracefully. The existing ID is used. But when the schema is different, the registry applies schema compatibility checks, and if they pass it updates the schema with a new ID. This patch first looks up the schema name in the registry, then generates its own schema and attemps to push it. This allows us to generate informational messages about whether the schema is new, existing, or updated/evolved.
If enabled, a list of default locations and variable names is checked. The file location can be overridden.
The patch clarifies that Python3 and munge development libraries are dependencies. This patch also adds a section that points to pre-built containers.
There is a compile error when built with -Wall -Werror. The host in the format is a static string that is 100 bytes. That with the other elements of the format couuld exceed 128 resulting in truncated output.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.