Repository Upload
This commit is contained in:
parent
5b649b50c0
commit
c716c99f18
134
README.md
134
README.md
@ -1,2 +1,134 @@
|
||||
# gitlab-ci-vars-setter
|
||||
# set-gitlab-vars
|
||||
|
||||
Universal script for setting GitLab CI/CD variables via API.
|
||||
|
||||
## Features
|
||||
- Set variables in bulk from a YAML file or individually via CLI arguments
|
||||
- Supports all GitLab variable options: environment scope, variable type (env_var/file), protected, raw, masked, masked_and_hidden, and description
|
||||
- Handles both creation and update of variables (idempotent)
|
||||
- Supports multiline and special characters for file and env_var types
|
||||
- YAML input supports full variable configuration, including descriptions
|
||||
- CLI input supports quick single variable setting
|
||||
|
||||
## Requirements
|
||||
- Bash (Linux/macOS)
|
||||
- `curl`
|
||||
- `yq` (for YAML input, see [yq installation](https://github.com/mikefarah/yq/#install))
|
||||
|
||||
## Usage
|
||||
|
||||
### 1. Bulk from YAML file
|
||||
```sh
|
||||
export GITLAB_URL="https://gitlab.example.com"
|
||||
export PROJECT=1 # or export PROJECT=owner_or_group/repo
|
||||
export GITLAB_TOKEN=... # token with api scope
|
||||
./set-gitlab-vars path/to/file.yml
|
||||
```
|
||||
|
||||
#### YAML file structure
|
||||
```yaml
|
||||
MY_SECRET:
|
||||
value: "supervalue" # required
|
||||
environment_scope: "Dev" # optional
|
||||
variable_type: "file" # optional (default: env_var)
|
||||
protected: true # optional (default: false)
|
||||
raw: false # optional (default: true)
|
||||
masked: false # optional (default: false)
|
||||
masked_and_hidden: false # optional (default: false)
|
||||
description: "Secret for Dev" # optional (default: null)
|
||||
```
|
||||
|
||||
### 2. Single variable via arguments
|
||||
```sh
|
||||
./set-gitlab-vars VAR_NAME VAR_VALUE [ENV_SCOPE] [VAR_TYPE] [PROTECTED] [RAW] [MASKED] [MASKED_AND_HIDDEN]
|
||||
# Example:
|
||||
./set-gitlab-vars MY_SECRET "supervalue" "Production" "file" true true false false
|
||||
```
|
||||
- Only `VAR_NAME` and `VAR_VALUE` are required. Other options are optional and default as in YAML.
|
||||
- `description` is only supported via YAML.
|
||||
|
||||
### 3. Help
|
||||
```sh
|
||||
./set-gitlab-vars --help
|
||||
```
|
||||
|
||||
## Required Environment Variables
|
||||
- `GITLAB_URL` — GitLab address (e.g. `https://gitlab.example.com`)
|
||||
- `PROJECT` — project ID or path (e.g. `1` or `owner_or_group/repo`)
|
||||
- `GITLAB_TOKEN` — token with api scope
|
||||
|
||||
## Notes
|
||||
- All boolean options (protected, raw, masked, masked_and_hidden) default to false except raw (default: true).
|
||||
- For file variables, multiline and special characters are supported.
|
||||
- For env_var variables, multiline values are supported via form-data.
|
||||
- All options are supported both in YAML and via CLI (except description, which is YAML only).
|
||||
|
||||
## Error Handling
|
||||
- The script will print errors and exit if required environment variables are missing or if the GitLab API returns an error.
|
||||
- If a variable already exists, it will be updated instead of created.
|
||||
|
||||
## JSON Schema for YAML Validation
|
||||
|
||||
A JSON schema file `gitlab-vars-schema.json` is included in this repository. You can use it to validate your YAML variable files for correctness and completeness before applying them with `set-gitlab-vars`.
|
||||
|
||||
### Usage Example (with `yamllint` and `ajv`)
|
||||
|
||||
1. Convert your YAML to JSON (if needed):
|
||||
```sh
|
||||
yq -o=json path/to/file.yml > file.json
|
||||
```
|
||||
2. Validate with `ajv`:
|
||||
```sh
|
||||
ajv validate -s gitlab-vars-schema.json -d file.json
|
||||
```
|
||||
|
||||
- The schema ensures all required fields and types are correct for each variable.
|
||||
- This helps catch errors early and maintain consistency in your CI/CD variable definitions.
|
||||
|
||||
For more details on the schema, see the `gitlab-vars-schema.json` file.
|
||||
|
||||
## Yamale Schema Validation
|
||||
|
||||
A Yamale schema (`gitlab-vars.yamale`) is provided for validating your GitLab CI/CD variables YAML files.
|
||||
|
||||
### Example Usage
|
||||
|
||||
1. Install Yamale (if not already installed):
|
||||
```sh
|
||||
pip install yamale
|
||||
```
|
||||
|
||||
2. Validate your YAML file:
|
||||
```sh
|
||||
yamale -s gitlab-vars.yamale path/to/your-vars.yml
|
||||
```
|
||||
|
||||
- Each top-level key in your YAML file should be a variable name.
|
||||
- The value for each variable is a mapping with the following fields:
|
||||
- `value` (string, required)
|
||||
- `environment_scope` (string, optional)
|
||||
- `variable_type` (string, optional)
|
||||
- `protected` (boolean, optional)
|
||||
- `raw` (boolean, optional)
|
||||
- `masked` (boolean, optional)
|
||||
- `masked_and_hidden` (boolean, optional)
|
||||
- `description` (string or null, optional)
|
||||
|
||||
#### Example YAML file
|
||||
```yaml
|
||||
MY_SECRET:
|
||||
value: "supervalue"
|
||||
environment_scope: "Dev"
|
||||
variable_type: "file"
|
||||
protected: true
|
||||
raw: false
|
||||
masked: false
|
||||
masked_and_hidden: false
|
||||
description: "Secret for Dev"
|
||||
```
|
||||
|
||||
For more details, see the `gitlab-vars.yamale` file.
|
||||
|
||||
## License
|
||||
See [LICENSE](LICENSE).
|
||||
|
||||
|
||||
23
gitlab-vars-schema.json
Normal file
23
gitlab-vars-schema.json
Normal file
@ -0,0 +1,23 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "GitLab CI/CD Variables File Schema",
|
||||
"type": "object",
|
||||
"patternProperties": {
|
||||
".+": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"value": { "type": "string" },
|
||||
"environment_scope": { "type": "string" },
|
||||
"variable_type": { "type": "string" },
|
||||
"protected": { "type": "boolean" },
|
||||
"raw": { "type": "boolean" },
|
||||
"masked": { "type": "boolean" },
|
||||
"masked_and_hidden": { "type": "boolean" },
|
||||
"description": { "type": ["string", "null"] }
|
||||
},
|
||||
"required": ["value"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
14
gitlab-vars.yamale
Normal file
14
gitlab-vars.yamale
Normal file
@ -0,0 +1,14 @@
|
||||
# Yamale schema for GitLab CI/CD Variables YAML file
|
||||
---
|
||||
# Each top-level key is a variable name (string)
|
||||
# The value is a mapping with the following fields:
|
||||
|
||||
<var_name>:
|
||||
value: str(required=True)
|
||||
environment_scope: str(required=False)
|
||||
variable_type: str(required=False)
|
||||
protected: bool(required=False)
|
||||
raw: bool(required=False)
|
||||
masked: bool(required=False)
|
||||
masked_and_hidden: bool(required=False)
|
||||
description: str(required=False, null=True)
|
||||
229
set-gitlab-vars
Executable file
229
set-gitlab-vars
Executable file
@ -0,0 +1,229 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
# Universal script for setting GitLab CI/CD variables via API
|
||||
|
||||
# Print help and exit if --help or -h is passed, or no arguments
|
||||
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" || $# -eq 0 ]]; then
|
||||
cat <<EOF
|
||||
set-gitlab-vars — universal script for setting GitLab CI/CD variables via API
|
||||
|
||||
Usage:
|
||||
1. Bulk from YAML file:
|
||||
export GITLAB_URL="https://gitlab.example.com"
|
||||
export PROJECT=4 # or export PROJECT=owner_or_group/repo
|
||||
export GITLAB_TOKEN=...
|
||||
set-gitlab-vars path/to/file.yml
|
||||
|
||||
2. Single variable via arguments:
|
||||
set-gitlab-vars VAR_NAME VAR_VALUE [ENV_SCOPE] [VAR_TYPE] [PROTECTED] [RAW] [MASKED] [MASKED_AND_HIDDEN]
|
||||
# Example:
|
||||
set-gitlab-vars MY_SECRET "supervalue" "Production" "file" true true false false
|
||||
|
||||
3. Help:
|
||||
set-gitlab-vars --help
|
||||
|
||||
YAML file structure:
|
||||
VAR_NAME:
|
||||
value: "..." # required
|
||||
environment_scope: "Dev" # optional
|
||||
variable_type: "env_var" # or "file", optional (default: env_var)
|
||||
protected: false # optional (default: false)
|
||||
raw: true # optional (default: true)
|
||||
masked: false # optional (default: false)
|
||||
masked_and_hidden: false # optional (default: false)
|
||||
description: "..." # optional (default: null)
|
||||
|
||||
Required environment variables:
|
||||
GITLAB_URL — GitLab address
|
||||
PROJECT — project ID or path (e.g. 4 or owner_or_group/repo)
|
||||
GITLAB_TOKEN — token with api scope
|
||||
|
||||
Notes:
|
||||
- All boolean options (protected, raw, masked, masked_and_hidden) default to false except raw (default: true).
|
||||
- description is only supported when using YAML input.
|
||||
- For file variables, multiline and special characters are supported.
|
||||
- For env_var variables, multiline values are supported via form-data.
|
||||
- All options are supported both in YAML and via CLI (except description, which is YAML only).
|
||||
EOF
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Error/info helpers
|
||||
err() { echo "[ERROR] $*" >&2; }
|
||||
info() { echo "[INFO] $*"; }
|
||||
|
||||
# Check required environment variables
|
||||
if [[ -z "${GITLAB_URL:-}" ]]; then
|
||||
err "GITLAB_URL is not set. Export GitLab address, e.g.: export GITLAB_URL=\"https://gitlab.example.com\""
|
||||
exit 1
|
||||
fi
|
||||
if [[ -z "${PROJECT:-}" ]]; then
|
||||
err "PROJECT is not set. Export project ID or path, e.g.: export PROJECT=4 or export PROJECT=owner_or_group/repo"
|
||||
exit 1
|
||||
fi
|
||||
if [[ -z "${GITLAB_TOKEN:-}" ]]; then
|
||||
err "GITLAB_TOKEN is not set. Export access token, e.g.: export GITLAB_TOKEN=\"<your_token>\""
|
||||
exit 1
|
||||
fi
|
||||
|
||||
urlencode() {
|
||||
local LANG=C
|
||||
local length="${#1}"
|
||||
for (( i = 0; i < length; i++ )); do
|
||||
local c="${1:i:1}"
|
||||
case $c in
|
||||
[a-zA-Z0-9.~_-]) printf "$c" ;;
|
||||
*) printf '%%%02X' "'${c}" ;;
|
||||
esac
|
||||
done
|
||||
}
|
||||
|
||||
PROJECT_ENC="$(urlencode "$PROJECT")"
|
||||
|
||||
CURL_BASE=(curl -sS --header "PRIVATE-TOKEN: ${GITLAB_TOKEN}")
|
||||
|
||||
# JSON escaping for file variables without python3
|
||||
escape_json() {
|
||||
local s="$1"
|
||||
s="${s//\\/\\\\}" # \
|
||||
s="${s//\"/\\\"}" # "
|
||||
s="${s//$'\n'/\\n}" # newlines
|
||||
s="${s//$'\t'/\\t}" # tabs
|
||||
echo -n "$s"
|
||||
}
|
||||
|
||||
set_var() {
|
||||
VAR_NAME="$1"
|
||||
VAR_VALUE="$2"
|
||||
ENVIRONMENT_SCOPE="${3:-}"
|
||||
VARIABLE_TYPE="${4:-env_var}"
|
||||
PROTECTED="${5:-false}"
|
||||
RAW="${6:-true}"
|
||||
MASKED="${7:-false}"
|
||||
MASKED_AND_HIDDEN="${8:-false}"
|
||||
DESCRIPTION="${9:-null}"
|
||||
info "Setting variable: $VAR_NAME, scope: $ENVIRONMENT_SCOPE, type: $VARIABLE_TYPE, protected: $PROTECTED, raw: $RAW, masked: $MASKED, masked_and_hidden: $MASKED_AND_HIDDEN, description: $DESCRIPTION"
|
||||
|
||||
# Prepare request body and headers
|
||||
if [[ "$VARIABLE_TYPE" == "file" ]]; then
|
||||
# For file variables, use JSON for correct escaping (no python3)
|
||||
ESCAPED_VALUE=$(escape_json "$VAR_VALUE")
|
||||
JSON_BODY="{\"key\":\"$VAR_NAME\",\"value\":\"$ESCAPED_VALUE\",\"variable_type\":\"$VARIABLE_TYPE\",\"protected\":$PROTECTED,\"raw\":$RAW,\"masked\":$MASKED,\"masked_and_hidden\":$MASKED_AND_HIDDEN"
|
||||
if [[ "$DESCRIPTION" != "null" ]]; then
|
||||
JSON_BODY+=" ,\"description\":\"$DESCRIPTION\""
|
||||
fi
|
||||
if [[ -n "$ENVIRONMENT_SCOPE" ]]; then
|
||||
JSON_BODY+=" ,\"environment_scope\":\"$ENVIRONMENT_SCOPE\""
|
||||
fi
|
||||
JSON_BODY+="}"
|
||||
REQUEST_ARGS=(--header "Content-Type: application/json" --data "$JSON_BODY")
|
||||
else
|
||||
# For env_var, use form-data to allow multiline
|
||||
REQUEST_ARGS=(
|
||||
--form "key=$VAR_NAME"
|
||||
--form "value=$VAR_VALUE"
|
||||
--form "variable_type=$VARIABLE_TYPE"
|
||||
--form "protected=$PROTECTED"
|
||||
--form "raw=$RAW"
|
||||
--form "masked=$MASKED"
|
||||
--form "masked_and_hidden=$MASKED_AND_HIDDEN"
|
||||
)
|
||||
if [[ "$DESCRIPTION" != "null" ]]; then
|
||||
REQUEST_ARGS+=(--form "description=$DESCRIPTION")
|
||||
fi
|
||||
if [[ -n "$ENVIRONMENT_SCOPE" ]]; then
|
||||
REQUEST_ARGS+=(--form "environment_scope=$ENVIRONMENT_SCOPE")
|
||||
fi
|
||||
fi
|
||||
|
||||
# Try to create variable
|
||||
RESPONSE_BODY=$(mktemp)
|
||||
RESPONSE_CODE=$("${CURL_BASE[@]}" -o "$RESPONSE_BODY" -w "%{http_code}" \
|
||||
--request POST "$GITLAB_URL/api/v4/projects/$PROJECT_ENC/variables" \
|
||||
"${REQUEST_ARGS[@]}")
|
||||
if [[ "$RESPONSE_CODE" == "201" ]]; then
|
||||
info "Variable $VAR_NAME created successfully."
|
||||
rm -f "$RESPONSE_BODY"
|
||||
return 0
|
||||
elif [[ "$RESPONSE_CODE" == "409" || "$RESPONSE_CODE" == "400" && $(grep -q 'has already been taken' "$RESPONSE_BODY" && echo 1) == 1 ]]; then
|
||||
info "Variable already exists, trying to update..."
|
||||
# Update variable
|
||||
UPDATE_URL="$GITLAB_URL/api/v4/projects/$PROJECT_ENC/variables/$VAR_NAME"
|
||||
if [[ -n "$ENVIRONMENT_SCOPE" ]]; then
|
||||
UPDATE_URL+="?filter%5Benvironment_scope%5D=$ENVIRONMENT_SCOPE"
|
||||
fi
|
||||
UPDATE_BODY=$(mktemp)
|
||||
if [[ "$VARIABLE_TYPE" == "file" ]]; then
|
||||
UPDATE_ARGS=(--header "Content-Type: application/json" --data "$JSON_BODY")
|
||||
else
|
||||
UPDATE_ARGS=(
|
||||
--form "key=$VAR_NAME"
|
||||
--form "value=$VAR_VALUE"
|
||||
--form "variable_type=$VARIABLE_TYPE"
|
||||
--form "protected=$PROTECTED"
|
||||
--form "raw=true"
|
||||
)
|
||||
if [[ -n "$ENVIRONMENT_SCOPE" ]]; then
|
||||
UPDATE_ARGS+=(--form "environment_scope=$ENVIRONMENT_SCOPE")
|
||||
fi
|
||||
fi
|
||||
RESPONSE2=$("${CURL_BASE[@]}" -o "$UPDATE_BODY" -w "%{http_code}" \
|
||||
--request PUT "$UPDATE_URL" "${UPDATE_ARGS[@]}")
|
||||
if [[ "$RESPONSE2" == "200" ]]; then
|
||||
info "Variable $VAR_NAME updated."
|
||||
rm -f "$RESPONSE_BODY" "$UPDATE_BODY"
|
||||
return 0
|
||||
else
|
||||
err "Failed to update variable $VAR_NAME (HTTP $RESPONSE2). GitLab API response: $(cat "$UPDATE_BODY")"
|
||||
rm -f "$RESPONSE_BODY" "$UPDATE_BODY"
|
||||
return 1
|
||||
fi
|
||||
elif [[ "$RESPONSE_CODE" == "400" ]]; then
|
||||
err "Failed to create variable $VAR_NAME (HTTP 400)."
|
||||
err "GitLab API response: $(cat "$RESPONSE_BODY")"
|
||||
rm -f "$RESPONSE_BODY"
|
||||
return 1
|
||||
else
|
||||
err "Failed to create variable $VAR_NAME (HTTP $RESPONSE_CODE)"
|
||||
rm -f "$RESPONSE_BODY"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# If a yaml file is passed as an argument, load variables from the file
|
||||
if [[ -n "${1:-}" && -f "$1" ]]; then
|
||||
if command -v yq >/dev/null 2>&1; then
|
||||
VARS_FILE="$1"
|
||||
VAR_NAMES=( $(yq 'keys | .[]' "$VARS_FILE") )
|
||||
for VAR in "${VAR_NAMES[@]}"; do
|
||||
[[ "$VAR" == "\$schema" ]] && continue
|
||||
VALUE=$(yq -r ".${VAR}.value" "$VARS_FILE")
|
||||
ENV_SCOPE=$(yq -r ".${VAR}.environment_scope // \"\"" "$VARS_FILE")
|
||||
VAR_TYPE=$(yq -r ".${VAR}.variable_type // \"env_var\"" "$VARS_FILE")
|
||||
PROTECTED=$(yq -r ".${VAR}.protected // false" "$VARS_FILE")
|
||||
RAW=$(yq -r ".${VAR}.raw // true" "$VARS_FILE")
|
||||
MASKED=$(yq -r ".${VAR}.masked // false" "$VARS_FILE")
|
||||
MASKED_AND_HIDDEN=$(yq -r ".${VAR}.masked_and_hidden // false" "$VARS_FILE")
|
||||
DESCRIPTION=$(yq -r ".${VAR}.description // null" "$VARS_FILE")
|
||||
set_var "$VAR" "$VALUE" "$ENV_SCOPE" "$VAR_TYPE" "$PROTECTED" "$RAW" "$MASKED" "$MASKED_AND_HIDDEN" "$DESCRIPTION"
|
||||
done
|
||||
exit 0
|
||||
else
|
||||
err "yq is required to work with yaml file (https://mikefarah.gitbook.io/yq/)"
|
||||
exit 2
|
||||
fi
|
||||
fi
|
||||
|
||||
# If 2-5 positional arguments are passed, set a single variable
|
||||
if [[ $# -ge 2 && $# -le 5 ]]; then
|
||||
VAR_NAME="$1"
|
||||
VAR_VALUE="$2"
|
||||
ENV_SCOPE="${3:-}"
|
||||
VAR_TYPE="${4:-env_var}"
|
||||
PROTECTED="${5:-false}"
|
||||
set_var "$VAR_NAME" "$VAR_VALUE" "$ENV_SCOPE" "$VAR_TYPE" "$PROTECTED"
|
||||
exit $?
|
||||
fi
|
||||
|
||||
err "Invalid arguments. Run with --help for usage."
|
||||
exit 1
|
||||
Loading…
x
Reference in New Issue
Block a user