I like the idea of pre-rendering templates with http://cheetahtemplate.org/ but it might be too hard to install and learn.

we just need simple pure python string substitution in config files that needs to be run before the files is rendered as jinja.

I don't think I understand :) What's the goal? To have a template be rendered... into another template, that is then rendered into a final state? So template1 > template2 > final?

Aside from not knowing why we need that, we should be able to do it with Jinja, or any other single templating engine. I'm not sure I get the case for needing different templating engine on top of it. Any double render would probably look pretty wonky though.

I thought an example would help. If you need to "skip" rendering some jinja, just surround it with some raw tags. For example,

{% raw %}{% skipped %}{% endraw %}

would compile to

{% skipped %}

the goal is template1 > template2 > final

using jinja on jinja would make the file looks something like this

{% raw %}my-ec2-config:
id: {% endraw %}{% id %}{% raw %}
key: {% endraw %}{% key %}{% raw %}{
keyname: default
securitygroup: ssh_open
private_key: /root/default.pem
service_l: amazonaws.com
endpoint: myendpoint.example.com:1138/services/Cloud
ssh_gateway: gateway.example.com
ssh_gateway_port: 22
ssh_gateway_username: root
ssh_gateway_private_key: /path/to/key.pem
ssh_gateway_password: ExamplePasswordHere
driver: ec2
userdata_file: /etc/salt/my-userdata-data
{% for n in range(1) %}

• minion-{{ n }}:
  grains:
  minion_hostname: minion-{{ n }}
  primary_role: load_balancer
  vm_size: 2048mb
  domain: test.local{% endraw %}

if we use something like "pure python string substitution" > jinja > final

the file looks something like this

my-ec2-config:
id: @id
key: @key
keyname: default
securitygroup: ssh_open
private_key: /root/default.pem
service_url: amazonaws.com
endpoint: myendpoint.example.com:1138/services/Cloud
ssh_gateway: gateway.example.com
ssh_gateway_port: 22
ssh_gateway_username: root
ssh_gateway_private_key: /path/to/key.pem
ssh_gateway_password: ExamplePasswordHere
driver: ec2
userdata_file: /etc/salt/my-userdata-data
{% for n in range(1) %}{% for n in range(1) %}
minion-{{ n }}:
grains:
minion_hostname: minion-{{ n }}
primary_role: load_balancer
vm_size: 2048mb
domain: test.local{% endraw %}

In your example, you have raw tags around what appears to just be strings, meaning they aren't doing anything. Bad example?

Also your vars should use {{, but that's not relevant really.

maybe it is a bad example. i added in a jinja loop to the the lower section of both.

Depending on the use, a simple string substitution might be best. I don't have a clear understanding of the use though. If there would need to be tons of raw tags with double jinja, it'd probably be too wonky for my tastes, but I don't have a problem with a few.

If you string substitution, you'd have to make sure there are no false positives for the string you're matching, of course. I don't know the situation well enough to know if that can be guaranteed.

the issues comes from dealing with the cloud config files
https://docs.saltstack.com/en/latest/topics/cloud/

the main files are

cloud
cloud.providers
cloud.profiles
map

Saltstack expect these files to have arbitrary jinja in them and saltstack expects them to have plain text secrets. Rambo needs to include a stock set of templates with json but without secrets. We need to copy the template (with arbitrary jinja in it) from the library to a location on disk that git ignores, then we need to inject the secret keys/tokens into the file.

This encoding might work

id: [[ id ]]
key: [[ @key ]]