Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow for conf.d like configuration #7950

Open
Manuelraa opened this issue Jun 20, 2023 · 6 comments
Open

Allow for conf.d like configuration #7950

Manuelraa opened this issue Jun 20, 2023 · 6 comments
Labels
area:config enhancement New feature or request

Comments

@Manuelraa
Copy link

Component(s)

cmd/configschema

Is your feature request related to a problem? Please describe.

For certain type of automatic deployments it is beneficial being able to split the config file into the main and per component configuration.
e.g. config.yaml and conf.d/apache.yaml

Describe the solution you'd like

It would be nice for --config parameter to support specifying a folder which it automatically would apply all *.yml/yaml config files from.

Describe alternatives you've considered

Alternative could be a giant jinja2 templated config.yaml which can get quite complex if used for lots of components in many mixes.
Alternative for splitting config file could be dynamically generating list of --config cli arguments on agent/systemd unit start based on a folder content. This sounds like a hacky solution.

Additional context

No response
@Manuelraa Manuelraa added the enhancement New feature or request label Jun 20, 2023
@github-actions
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@mx-psi mx-psi added the question Further information is requested label Jun 20, 2023
@mx-psi
Copy link
Member

mx-psi commented Jun 20, 2023

Hi @Manuelraa! You can provide your configuration in multiple files by using the ${file:/path/to/file} syntax. You can see an example here. Would that work for you?

@Manuelraa
Copy link
Author

I don't think that would cover my use case. We would need a feature which would allow us dynamically loading and merging configs from a folder.
./otelcol-contrib --config-file /etc/otelcol-contrib/config.yml --config-dir /etc/otelcol-contrib/conf.d

The long story/example

Assuming more traditional VM infrastructure with Ansible automation.
The automation split in a way that they are independent.

There is one ansible playbook/role which deploys otelcol and its main/global configuration. Deploying e.g. /etc/otelcol/config.yml

Now there is another ansible playbook/role which deployes a apache httpd instance on the host. This role should also provide the necessary configuration to collect metrics from it.
There can be multiple apache httpd instances per host.
Each one would deploy its own monitoring configuration defining their own receivers e.g. /etc/otelcol/conf.d/apache-instance1.yml, /etc/otelcol/conf.d/apache-instance2.yml, /etc/otelcol/conf.d/apache-instance3.yml

In case the logic would not be split up this like this the otelcol role would need to know about all instances running on the host and their information.
If it is split up the automation deploying the component can also deploy its monitoring configuration.

@mx-psi mx-psi removed the question Further information is requested label Jun 21, 2023
@mx-psi mx-psi transferred this issue from open-telemetry/opentelemetry-collector-contrib Jun 21, 2023
@mx-psi
Copy link
Member

mx-psi commented Jun 21, 2023

Alright, thanks for the details. I moved this to the opentelemetry-collector repository then, since this is a confmap feature.

@haoqixu
Copy link
Member

haoqixu commented Sep 13, 2023

It seems that globs support of file provider (#7177) can cover this use case.

@avesst
Copy link

avesst commented Dec 1, 2023

I don't think that would cover my use case. We would need a feature which would allow us dynamically loading and merging configs from a folder. ./otelcol-contrib --config-file /etc/otelcol-contrib/config.yml --config-dir /etc/otelcol-contrib/conf.d

The long story/example

Exact same use case here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area:config enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@avesst @mx-psi @Manuelraa @haoqixu