Repository Upload
This commit is contained in:
parent
5b649b50c0
commit
c716c99f18
134
README.md
134
README.md
@ -1,2 +1,134 @@
|
|||||||
# gitlab-ci-vars-setter
|
# set-gitlab-vars
|
||||||
|
|
||||||
|
Universal script for setting GitLab CI/CD variables via API.
|
||||||
|
|
||||||
|
## Features
|
||||||
|
- Set variables in bulk from a YAML file or individually via CLI arguments
|
||||||
|
- Supports all GitLab variable options: environment scope, variable type (env_var/file), protected, raw, masked, masked_and_hidden, and description
|
||||||
|
- Handles both creation and update of variables (idempotent)
|
||||||
|
- Supports multiline and special characters for file and env_var types
|
||||||
|
- YAML input supports full variable configuration, including descriptions
|
||||||
|
- CLI input supports quick single variable setting
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
- Bash (Linux/macOS)
|
||||||
|
- `curl`
|
||||||
|
- `yq` (for YAML input, see [yq installation](https://github.com/mikefarah/yq/#install))
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### 1. Bulk from YAML file
|
||||||
|
```sh
|
||||||
|
export GITLAB_URL="https://gitlab.example.com"
|
||||||
|
export PROJECT=1 # or export PROJECT=owner_or_group/repo
|
||||||
|
export GITLAB_TOKEN=... # token with api scope
|
||||||
|
./set-gitlab-vars path/to/file.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
#### YAML file structure
|
||||||
|
```yaml
|
||||||
|
MY_SECRET:
|
||||||
|
value: "supervalue" # required
|
||||||
|
environment_scope: "Dev" # optional
|
||||||
|
variable_type: "file" # optional (default: env_var)
|
||||||
|
protected: true # optional (default: false)
|
||||||
|
raw: false # optional (default: true)
|
||||||
|
masked: false # optional (default: false)
|
||||||
|
masked_and_hidden: false # optional (default: false)
|
||||||
|
description: "Secret for Dev" # optional (default: null)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Single variable via arguments
|
||||||
|
```sh
|
||||||
|
./set-gitlab-vars VAR_NAME VAR_VALUE [ENV_SCOPE] [VAR_TYPE] [PROTECTED] [RAW] [MASKED] [MASKED_AND_HIDDEN]
|
||||||
|
# Example:
|
||||||
|
./set-gitlab-vars MY_SECRET "supervalue" "Production" "file" true true false false
|
||||||
|
```
|
||||||
|
- Only `VAR_NAME` and `VAR_VALUE` are required. Other options are optional and default as in YAML.
|
||||||
|
- `description` is only supported via YAML.
|
||||||
|
|
||||||
|
### 3. Help
|
||||||
|
```sh
|
||||||
|
./set-gitlab-vars --help
|
||||||
|
```
|
||||||
|
|
||||||
|
## Required Environment Variables
|
||||||
|
- `GITLAB_URL` — GitLab address (e.g. `https://gitlab.example.com`)
|
||||||
|
- `PROJECT` — project ID or path (e.g. `1` or `owner_or_group/repo`)
|
||||||
|
- `GITLAB_TOKEN` — token with api scope
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
- All boolean options (protected, raw, masked, masked_and_hidden) default to false except raw (default: true).
|
||||||
|
- For file variables, multiline and special characters are supported.
|
||||||
|
- For env_var variables, multiline values are supported via form-data.
|
||||||
|
- All options are supported both in YAML and via CLI (except description, which is YAML only).
|
||||||
|
|
||||||
|
## Error Handling
|
||||||
|
- The script will print errors and exit if required environment variables are missing or if the GitLab API returns an error.
|
||||||
|
- If a variable already exists, it will be updated instead of created.
|
||||||
|
|
||||||
|
## JSON Schema for YAML Validation
|
||||||
|
|
||||||
|
A JSON schema file `gitlab-vars-schema.json` is included in this repository. You can use it to validate your YAML variable files for correctness and completeness before applying them with `set-gitlab-vars`.
|
||||||
|
|
||||||
|
### Usage Example (with `yamllint` and `ajv`)
|
||||||
|
|
||||||
|
1. Convert your YAML to JSON (if needed):
|
||||||
|
```sh
|
||||||
|
yq -o=json path/to/file.yml > file.json
|
||||||
|
```
|
||||||
|
2. Validate with `ajv`:
|
||||||
|
```sh
|
||||||
|
ajv validate -s gitlab-vars-schema.json -d file.json
|
||||||
|
```
|
||||||
|
|
||||||
|
- The schema ensures all required fields and types are correct for each variable.
|
||||||
|
- This helps catch errors early and maintain consistency in your CI/CD variable definitions.
|
||||||
|
|
||||||
|
For more details on the schema, see the `gitlab-vars-schema.json` file.
|
||||||
|
|
||||||
|
## Yamale Schema Validation
|
||||||
|
|
||||||
|
A Yamale schema (`gitlab-vars.yamale`) is provided for validating your GitLab CI/CD variables YAML files.
|
||||||
|
|
||||||
|
### Example Usage
|
||||||
|
|
||||||
|
1. Install Yamale (if not already installed):
|
||||||
|
```sh
|
||||||
|
pip install yamale
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Validate your YAML file:
|
||||||
|
```sh
|
||||||
|
yamale -s gitlab-vars.yamale path/to/your-vars.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
- Each top-level key in your YAML file should be a variable name.
|
||||||
|
- The value for each variable is a mapping with the following fields:
|
||||||
|
- `value` (string, required)
|
||||||
|
- `environment_scope` (string, optional)
|
||||||
|
- `variable_type` (string, optional)
|
||||||
|
- `protected` (boolean, optional)
|
||||||
|
- `raw` (boolean, optional)
|
||||||
|
- `masked` (boolean, optional)
|
||||||
|
- `masked_and_hidden` (boolean, optional)
|
||||||
|
- `description` (string or null, optional)
|
||||||
|
|
||||||
|
#### Example YAML file
|
||||||
|
```yaml
|
||||||
|
MY_SECRET:
|
||||||
|
value: "supervalue"
|
||||||
|
environment_scope: "Dev"
|
||||||
|
variable_type: "file"
|
||||||
|
protected: true
|
||||||
|
raw: false
|
||||||
|
masked: false
|
||||||
|
masked_and_hidden: false
|
||||||
|
description: "Secret for Dev"
|
||||||
|
```
|
||||||
|
|
||||||
|
For more details, see the `gitlab-vars.yamale` file.
|
||||||
|
|
||||||
|
## License
|
||||||
|
See [LICENSE](LICENSE).
|
||||||
|
|
||||||
|
|||||||
23
gitlab-vars-schema.json
Normal file
23
gitlab-vars-schema.json
Normal file
@ -0,0 +1,23 @@
|
|||||||
|
{
|
||||||
|
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||||
|
"title": "GitLab CI/CD Variables File Schema",
|
||||||
|
"type": "object",
|
||||||
|
"patternProperties": {
|
||||||
|
".+": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"value": { "type": "string" },
|
||||||
|
"environment_scope": { "type": "string" },
|
||||||
|
"variable_type": { "type": "string" },
|
||||||
|
"protected": { "type": "boolean" },
|
||||||
|
"raw": { "type": "boolean" },
|
||||||
|
"masked": { "type": "boolean" },
|
||||||
|
"masked_and_hidden": { "type": "boolean" },
|
||||||
|
"description": { "type": ["string", "null"] }
|
||||||
|
},
|
||||||
|
"required": ["value"],
|
||||||
|
"additionalProperties": false
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
}
|
||||||
14
gitlab-vars.yamale
Normal file
14
gitlab-vars.yamale
Normal file
@ -0,0 +1,14 @@
|
|||||||
|
# Yamale schema for GitLab CI/CD Variables YAML file
|
||||||
|
---
|
||||||
|
# Each top-level key is a variable name (string)
|
||||||
|
# The value is a mapping with the following fields:
|
||||||
|
|
||||||
|
<var_name>:
|
||||||
|
value: str(required=True)
|
||||||
|
environment_scope: str(required=False)
|
||||||
|
variable_type: str(required=False)
|
||||||
|
protected: bool(required=False)
|
||||||
|
raw: bool(required=False)
|
||||||
|
masked: bool(required=False)
|
||||||
|
masked_and_hidden: bool(required=False)
|
||||||
|
description: str(required=False, null=True)
|
||||||
229
set-gitlab-vars
Executable file
229
set-gitlab-vars
Executable file
@ -0,0 +1,229 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
set -euo pipefail
|
||||||
|
# Universal script for setting GitLab CI/CD variables via API
|
||||||
|
|
||||||
|
# Print help and exit if --help or -h is passed, or no arguments
|
||||||
|
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" || $# -eq 0 ]]; then
|
||||||
|
cat <<EOF
|
||||||
|
set-gitlab-vars — universal script for setting GitLab CI/CD variables via API
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
1. Bulk from YAML file:
|
||||||
|
export GITLAB_URL="https://gitlab.example.com"
|
||||||
|
export PROJECT=4 # or export PROJECT=owner_or_group/repo
|
||||||
|
export GITLAB_TOKEN=...
|
||||||
|
set-gitlab-vars path/to/file.yml
|
||||||
|
|
||||||
|
2. Single variable via arguments:
|
||||||
|
set-gitlab-vars VAR_NAME VAR_VALUE [ENV_SCOPE] [VAR_TYPE] [PROTECTED] [RAW] [MASKED] [MASKED_AND_HIDDEN]
|
||||||
|
# Example:
|
||||||
|
set-gitlab-vars MY_SECRET "supervalue" "Production" "file" true true false false
|
||||||
|
|
||||||
|
3. Help:
|
||||||
|
set-gitlab-vars --help
|
||||||
|
|
||||||
|
YAML file structure:
|
||||||
|
VAR_NAME:
|
||||||
|
value: "..." # required
|
||||||
|
environment_scope: "Dev" # optional
|
||||||
|
variable_type: "env_var" # or "file", optional (default: env_var)
|
||||||
|
protected: false # optional (default: false)
|
||||||
|
raw: true # optional (default: true)
|
||||||
|
masked: false # optional (default: false)
|
||||||
|
masked_and_hidden: false # optional (default: false)
|
||||||
|
description: "..." # optional (default: null)
|
||||||
|
|
||||||
|
Required environment variables:
|
||||||
|
GITLAB_URL — GitLab address
|
||||||
|
PROJECT — project ID or path (e.g. 4 or owner_or_group/repo)
|
||||||
|
GITLAB_TOKEN — token with api scope
|
||||||
|
|
||||||
|
Notes:
|
||||||
|
- All boolean options (protected, raw, masked, masked_and_hidden) default to false except raw (default: true).
|
||||||
|
- description is only supported when using YAML input.
|
||||||
|
- For file variables, multiline and special characters are supported.
|
||||||
|
- For env_var variables, multiline values are supported via form-data.
|
||||||
|
- All options are supported both in YAML and via CLI (except description, which is YAML only).
|
||||||
|
EOF
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Error/info helpers
|
||||||
|
err() { echo "[ERROR] $*" >&2; }
|
||||||
|
info() { echo "[INFO] $*"; }
|
||||||
|
|
||||||
|
# Check required environment variables
|
||||||
|
if [[ -z "${GITLAB_URL:-}" ]]; then
|
||||||
|
err "GITLAB_URL is not set. Export GitLab address, e.g.: export GITLAB_URL=\"https://gitlab.example.com\""
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if [[ -z "${PROJECT:-}" ]]; then
|
||||||
|
err "PROJECT is not set. Export project ID or path, e.g.: export PROJECT=4 or export PROJECT=owner_or_group/repo"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if [[ -z "${GITLAB_TOKEN:-}" ]]; then
|
||||||
|
err "GITLAB_TOKEN is not set. Export access token, e.g.: export GITLAB_TOKEN=\"<your_token>\""
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
urlencode() {
|
||||||
|
local LANG=C
|
||||||
|
local length="${#1}"
|
||||||
|
for (( i = 0; i < length; i++ )); do
|
||||||
|
local c="${1:i:1}"
|
||||||
|
case $c in
|
||||||
|
[a-zA-Z0-9.~_-]) printf "$c" ;;
|
||||||
|
*) printf '%%%02X' "'${c}" ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
PROJECT_ENC="$(urlencode "$PROJECT")"
|
||||||
|
|
||||||
|
CURL_BASE=(curl -sS --header "PRIVATE-TOKEN: ${GITLAB_TOKEN}")
|
||||||
|
|
||||||
|
# JSON escaping for file variables without python3
|
||||||
|
escape_json() {
|
||||||
|
local s="$1"
|
||||||
|
s="${s//\\/\\\\}" # \
|
||||||
|
s="${s//\"/\\\"}" # "
|
||||||
|
s="${s//$'\n'/\\n}" # newlines
|
||||||
|
s="${s//$'\t'/\\t}" # tabs
|
||||||
|
echo -n "$s"
|
||||||
|
}
|
||||||
|
|
||||||
|
set_var() {
|
||||||
|
VAR_NAME="$1"
|
||||||
|
VAR_VALUE="$2"
|
||||||
|
ENVIRONMENT_SCOPE="${3:-}"
|
||||||
|
VARIABLE_TYPE="${4:-env_var}"
|
||||||
|
PROTECTED="${5:-false}"
|
||||||
|
RAW="${6:-true}"
|
||||||
|
MASKED="${7:-false}"
|
||||||
|
MASKED_AND_HIDDEN="${8:-false}"
|
||||||
|
DESCRIPTION="${9:-null}"
|
||||||
|
info "Setting variable: $VAR_NAME, scope: $ENVIRONMENT_SCOPE, type: $VARIABLE_TYPE, protected: $PROTECTED, raw: $RAW, masked: $MASKED, masked_and_hidden: $MASKED_AND_HIDDEN, description: $DESCRIPTION"
|
||||||
|
|
||||||
|
# Prepare request body and headers
|
||||||
|
if [[ "$VARIABLE_TYPE" == "file" ]]; then
|
||||||
|
# For file variables, use JSON for correct escaping (no python3)
|
||||||
|
ESCAPED_VALUE=$(escape_json "$VAR_VALUE")
|
||||||
|
JSON_BODY="{\"key\":\"$VAR_NAME\",\"value\":\"$ESCAPED_VALUE\",\"variable_type\":\"$VARIABLE_TYPE\",\"protected\":$PROTECTED,\"raw\":$RAW,\"masked\":$MASKED,\"masked_and_hidden\":$MASKED_AND_HIDDEN"
|
||||||
|
if [[ "$DESCRIPTION" != "null" ]]; then
|
||||||
|
JSON_BODY+=" ,\"description\":\"$DESCRIPTION\""
|
||||||
|
fi
|
||||||
|
if [[ -n "$ENVIRONMENT_SCOPE" ]]; then
|
||||||
|
JSON_BODY+=" ,\"environment_scope\":\"$ENVIRONMENT_SCOPE\""
|
||||||
|
fi
|
||||||
|
JSON_BODY+="}"
|
||||||
|
REQUEST_ARGS=(--header "Content-Type: application/json" --data "$JSON_BODY")
|
||||||
|
else
|
||||||
|
# For env_var, use form-data to allow multiline
|
||||||
|
REQUEST_ARGS=(
|
||||||
|
--form "key=$VAR_NAME"
|
||||||
|
--form "value=$VAR_VALUE"
|
||||||
|
--form "variable_type=$VARIABLE_TYPE"
|
||||||
|
--form "protected=$PROTECTED"
|
||||||
|
--form "raw=$RAW"
|
||||||
|
--form "masked=$MASKED"
|
||||||
|
--form "masked_and_hidden=$MASKED_AND_HIDDEN"
|
||||||
|
)
|
||||||
|
if [[ "$DESCRIPTION" != "null" ]]; then
|
||||||
|
REQUEST_ARGS+=(--form "description=$DESCRIPTION")
|
||||||
|
fi
|
||||||
|
if [[ -n "$ENVIRONMENT_SCOPE" ]]; then
|
||||||
|
REQUEST_ARGS+=(--form "environment_scope=$ENVIRONMENT_SCOPE")
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Try to create variable
|
||||||
|
RESPONSE_BODY=$(mktemp)
|
||||||
|
RESPONSE_CODE=$("${CURL_BASE[@]}" -o "$RESPONSE_BODY" -w "%{http_code}" \
|
||||||
|
--request POST "$GITLAB_URL/api/v4/projects/$PROJECT_ENC/variables" \
|
||||||
|
"${REQUEST_ARGS[@]}")
|
||||||
|
if [[ "$RESPONSE_CODE" == "201" ]]; then
|
||||||
|
info "Variable $VAR_NAME created successfully."
|
||||||
|
rm -f "$RESPONSE_BODY"
|
||||||
|
return 0
|
||||||
|
elif [[ "$RESPONSE_CODE" == "409" || "$RESPONSE_CODE" == "400" && $(grep -q 'has already been taken' "$RESPONSE_BODY" && echo 1) == 1 ]]; then
|
||||||
|
info "Variable already exists, trying to update..."
|
||||||
|
# Update variable
|
||||||
|
UPDATE_URL="$GITLAB_URL/api/v4/projects/$PROJECT_ENC/variables/$VAR_NAME"
|
||||||
|
if [[ -n "$ENVIRONMENT_SCOPE" ]]; then
|
||||||
|
UPDATE_URL+="?filter%5Benvironment_scope%5D=$ENVIRONMENT_SCOPE"
|
||||||
|
fi
|
||||||
|
UPDATE_BODY=$(mktemp)
|
||||||
|
if [[ "$VARIABLE_TYPE" == "file" ]]; then
|
||||||
|
UPDATE_ARGS=(--header "Content-Type: application/json" --data "$JSON_BODY")
|
||||||
|
else
|
||||||
|
UPDATE_ARGS=(
|
||||||
|
--form "key=$VAR_NAME"
|
||||||
|
--form "value=$VAR_VALUE"
|
||||||
|
--form "variable_type=$VARIABLE_TYPE"
|
||||||
|
--form "protected=$PROTECTED"
|
||||||
|
--form "raw=true"
|
||||||
|
)
|
||||||
|
if [[ -n "$ENVIRONMENT_SCOPE" ]]; then
|
||||||
|
UPDATE_ARGS+=(--form "environment_scope=$ENVIRONMENT_SCOPE")
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
RESPONSE2=$("${CURL_BASE[@]}" -o "$UPDATE_BODY" -w "%{http_code}" \
|
||||||
|
--request PUT "$UPDATE_URL" "${UPDATE_ARGS[@]}")
|
||||||
|
if [[ "$RESPONSE2" == "200" ]]; then
|
||||||
|
info "Variable $VAR_NAME updated."
|
||||||
|
rm -f "$RESPONSE_BODY" "$UPDATE_BODY"
|
||||||
|
return 0
|
||||||
|
else
|
||||||
|
err "Failed to update variable $VAR_NAME (HTTP $RESPONSE2). GitLab API response: $(cat "$UPDATE_BODY")"
|
||||||
|
rm -f "$RESPONSE_BODY" "$UPDATE_BODY"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
elif [[ "$RESPONSE_CODE" == "400" ]]; then
|
||||||
|
err "Failed to create variable $VAR_NAME (HTTP 400)."
|
||||||
|
err "GitLab API response: $(cat "$RESPONSE_BODY")"
|
||||||
|
rm -f "$RESPONSE_BODY"
|
||||||
|
return 1
|
||||||
|
else
|
||||||
|
err "Failed to create variable $VAR_NAME (HTTP $RESPONSE_CODE)"
|
||||||
|
rm -f "$RESPONSE_BODY"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# If a yaml file is passed as an argument, load variables from the file
|
||||||
|
if [[ -n "${1:-}" && -f "$1" ]]; then
|
||||||
|
if command -v yq >/dev/null 2>&1; then
|
||||||
|
VARS_FILE="$1"
|
||||||
|
VAR_NAMES=( $(yq 'keys | .[]' "$VARS_FILE") )
|
||||||
|
for VAR in "${VAR_NAMES[@]}"; do
|
||||||
|
[[ "$VAR" == "\$schema" ]] && continue
|
||||||
|
VALUE=$(yq -r ".${VAR}.value" "$VARS_FILE")
|
||||||
|
ENV_SCOPE=$(yq -r ".${VAR}.environment_scope // \"\"" "$VARS_FILE")
|
||||||
|
VAR_TYPE=$(yq -r ".${VAR}.variable_type // \"env_var\"" "$VARS_FILE")
|
||||||
|
PROTECTED=$(yq -r ".${VAR}.protected // false" "$VARS_FILE")
|
||||||
|
RAW=$(yq -r ".${VAR}.raw // true" "$VARS_FILE")
|
||||||
|
MASKED=$(yq -r ".${VAR}.masked // false" "$VARS_FILE")
|
||||||
|
MASKED_AND_HIDDEN=$(yq -r ".${VAR}.masked_and_hidden // false" "$VARS_FILE")
|
||||||
|
DESCRIPTION=$(yq -r ".${VAR}.description // null" "$VARS_FILE")
|
||||||
|
set_var "$VAR" "$VALUE" "$ENV_SCOPE" "$VAR_TYPE" "$PROTECTED" "$RAW" "$MASKED" "$MASKED_AND_HIDDEN" "$DESCRIPTION"
|
||||||
|
done
|
||||||
|
exit 0
|
||||||
|
else
|
||||||
|
err "yq is required to work with yaml file (https://mikefarah.gitbook.io/yq/)"
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# If 2-5 positional arguments are passed, set a single variable
|
||||||
|
if [[ $# -ge 2 && $# -le 5 ]]; then
|
||||||
|
VAR_NAME="$1"
|
||||||
|
VAR_VALUE="$2"
|
||||||
|
ENV_SCOPE="${3:-}"
|
||||||
|
VAR_TYPE="${4:-env_var}"
|
||||||
|
PROTECTED="${5:-false}"
|
||||||
|
set_var "$VAR_NAME" "$VAR_VALUE" "$ENV_SCOPE" "$VAR_TYPE" "$PROTECTED"
|
||||||
|
exit $?
|
||||||
|
fi
|
||||||
|
|
||||||
|
err "Invalid arguments. Run with --help for usage."
|
||||||
|
exit 1
|
||||||
Loading…
x
Reference in New Issue
Block a user