Skip to content

Commit

Permalink
Update YAML permissions in configuration
Browse files Browse the repository at this point in the history 
Add cache_ip option to aggregators->peers
Update documentation
  • Loading branch information
@nick-enoent
nick-enoent committed Dec 13, 2024
1 parent 39f72b8 commit 2536905
Show file tree
Hide file tree
Showing 2 changed files with 45 additions and 34 deletions.
30 changes: 15 additions & 15 deletions ldms/man/ldmsd_yaml_parser.man
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ The name of the LDMS daemon to configure.

The LDMS YAML configuration syntax allows defining a single LMDSDs configuration up to an entire LDMS cluster configuration in a single file.
.br
LDMS YAML cluster configuration is organized into “groups” of dictionaries with relevant configurations for each group. The top level dictionaries are “daemons”, “aggregators”, “samplers”, “plugins”, and “stores”.
LDMS YAML cluster configuration is organized into "groups" of dictionaries with relevant configurations for each group. The top level dictionaries are "daemons", "aggregators", "samplers", "plugins", and "stores".
.br
Time intervals are defined as a unit string, or an integer in microseconds.
.br
Expand All @@ -48,18 +48,14 @@ A unit string is one of the followings:
d -- days
.br
.PP
Permissions are defined in a string linux file system format, with a dash separating users, groups, and global. A string of an octal number is also accepted. e.g. "0777"
Permissions are defined in a unix-like permissions format, with a dash indicating a lack of permission. Executable permisson is not supported by "perm" and should always be "-". e.g. "rw-r-----"
.br
A string of an octal number is also accepted. e.g. "0640"
.br
"r" represents read permissions.
.br
"w" or "+" represent write permissions.
.br
"x" represents execute permissions.
.br
If user, group, or global is left blank, the configuration will assume there are no permissions for this group. e.g. "rw-r-", denotes that the user has read/write permissons, the group has read permissions, and global has no permissons.
.br
The permissions syntax expects there will always be 3 "-" to seperate users, groups, and global.
.br
.PP
Several of the keys in this configuration syntax utilize the hostlist format.
.br
Expand Down Expand Up @@ -156,7 +152,7 @@ The name of the stream.
Regular expression matching producers to subscribe to the stream.

.SS peers
List of dictionaries containing producer configurations. This is an alternative method to configuring producers than using prdcr_listen
List of dictionaries containing producer configurations. This is an alternative method to configuring producers than using prdcr_listen. Producers defined in the "peers" section are as evenly distributed as possible amongst the "aggregators" defined in the parent directory. e.g. If there are 2 aggregators, and 4 producers, each aggregator will be assigned 2 producers in the configuration.
.TP
.BR daemons
.br
Expand All @@ -176,7 +172,11 @@ Producer type. Either active or passive. passive is being deprecated.
.TP
.BR [perm]
.br
The permissions to modify the producer in the future.
The permissions to modify the producer in the future. String of octal number or unix-like permissions format. e.g. "rw-r--r--"
.TP
.BR [cache_ip]
.br
True/False boolean. True will cache the IP address after the first successful resolution (default). False will resolve the hostname at prdcr_add and at every connection attempt.
.TP
.BR updaters
.br
Expand All @@ -203,7 +203,7 @@ Offset for synchronized aggregation. Optional. Unit string format.
.TP
.BR [perm]
.br
The permissions that allow modification of an updater in the future.
The permissions that allow modification of an updater in the future. String of octal number or unix-like permissions format. e.g. "rw-r--r--"
.TP
.BR [producers]
.br
Expand Down Expand Up @@ -268,7 +268,7 @@ Offset for synchronized aggregation. Optional. Unit string format.
.TP
.BR [perm]
.br
The permissions to modify the producer in the future.
The permissions to modify the producer in the future. String of octal number or unix-like permissions format. e.g. "rw-r--r--"
.TP
.BR [producers]
.br
Expand Down Expand Up @@ -320,7 +320,7 @@ String of port(s) in hostlist format of the aggregator daemons that the sampler
The interval at which the sampler will attempt to reconnect to a disconnected advertiser. Float followed by a unit string.
.TP
.BR [perm]
The permissions in order to modify the advertiser in the future. String linux file system format. e.g. "rw-r-"
The permissions in order to modify the advertiser in the future. String of octal number or unix-like permissions format. e.g. "rw-r--r--"
.TP
.BR [auth]
.br
Expand Down Expand Up @@ -358,7 +358,7 @@ Name of a storage plugin that matches a key of a plugin defined in the top level
.TP
.BR [perm]
.br
The permissions of who can modify the storage plugin in the future. Linux file system format e.g. "rw-r-r"
The permissions of who can modify the storage plugin in the future. String of octal number or unix-like permissions format. e.g. "rw-r--r--"
.TP
.BR [decomposition]
.br
Expand Down Expand Up @@ -408,7 +408,7 @@ Name of the metric set to use.
.TP
.BR [perm]
.br
Access permissions for the metric set within the container. e.g. "0440"
Access permissions for the metric set within the container. String of octal number or unix-like permissions format. e.g. "rw-r--r--"
.TP
.BR [component_id]
.br
Expand Down
49 changes: 30 additions & 19 deletions ldms/python/ldmsd/parser_util.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -169,27 +169,35 @@ def parse_yaml_bool(bool_):
return False

def perm_handler(perm_str):
if perm_str is None or type(perm_str) is int:
if perm_str is None:
return perm_str
nperm = "0"
perm_str = perm_str.split('-')
if len(perm_str) == 1:
if type(perm_str) is not str:
raise TypeError(f'Error: YAML "perms" value must be a string')
nperm = '0'
if len(perm_str.split('-')) > 1:
if len(perm_str) != 9:
raise ValueError(f'Error: Parsing YAML permisson string "{perm_str}"\n'
f'When using unix-like vernacular to configure permissions you must account for read/write in user/group/global\n'
f'Allowed format: (r|-)(w|-)-(r|-)(w|-)-(r|-)(w|-)-')
i = 0
x = 0
for ch in perm_str:
if i == 2:
nperm += str(x)
x = 0
i = 0
continue
if ch == 'r':
x += 4
if ch == 'w' or ch == '+':
x += 2
i += 1
else:
try:
z = int(perm_str[0])
return perm_str[0]
z = int(perm_str)
return perm_str
except:
raise ValueError(f'Error: permission {perm_str[0]} is not a valid value')
if len(perm_str) > 3:
raise ValueError(f'There are only 3 definable characteristics of linux permissions; users, group, and global. Please modify {perm_str} to reflect this\n')
for uog in perm_str:
x = 0
if 'r' in uog:
x += 4
if 'w' in uog or '+' in uog:
x += 2
if 'x' in uog:
x += 1
nperm += str(x)
raise ValueError(f'Error: permission {perm_str} is not a valid value')
return nperm

class YamlCfg(object):
Expand Down Expand Up @@ -462,6 +470,7 @@ def build_producers(self, config):
reconnect = check_intrvl_str(prod['reconnect'])
perm = check_opt('perm', prod)
perm = perm_handler(perm)
cache_ip = check_opt('cache_ip', prod)
ports_per_dmn = len(endpoints) / len(smplr_dmns)
ppd = ports_per_dmn
try:
Expand All @@ -482,6 +491,7 @@ def build_producers(self, config):
'group' : group,
'reconnect' : reconnect,
'perm' : perm,
'cache_ip' : cache_ip,
'updaters' : upd_spec
}
producers[group][endpoint] = prod
Expand Down Expand Up @@ -810,12 +820,14 @@ def write_producers(self, dstr, group_name, dmn, auth_list):
perm = check_opt('perm', producer)
ptype = producer['type']
reconnect = producer['reconnect']
cache_ip = check_opt('cache_ip', producer)
dstr += f'prdcr_add name={pname} '\
f'host={hostname} '\
f'port={port} '\
f'xprt={xprt} '\
f'type={ptype} '\
f'reconnect={reconnect}'
dstr = self.write_opt_attr(dstr, 'cache_ip', cache_ip, endline=False)
dstr = self.write_opt_attr(dstr, 'perm', perm, endline=False)
dstr = self.write_opt_attr(dstr, 'auth', auth)
last_sampler = pname
Expand All @@ -828,7 +840,6 @@ def write_producers(self, dstr, group_name, dmn, auth_list):
def write_options(self, dstr, grp, dname):
if 'cli_opt' not in self.daemons[grp][dname]:
return dstr
self.daemons[grp][dname]['cli_opt']
cli_opt = self.daemons[grp][dname]['cli_opt']
for opt in cli_opt:
if type(cli_opt[opt]) is dict:
Expand Down

0 comments on commit 2536905

Please sign in to comment.