GitLab CI Jobs and Stages in GitLab CI Explained

In this article we are going to cover what is Jobs in GitLab CI | GitLab CI Jobs and Stages in GitLab CI ?

What is Jobs in GitLab CI | GitLab CI Jobs ?

In the YAML file, the main component is Jobs. You specify instructions for GitLab CI/CD means structure and unlimited number of jobs you have to define. Each job must have a unique name that cannot be the keywords in GitLab CI/CD. A job is defined by a list of parameters; the parameters define the job behavior. When you commit the YAML file to your repository then runner runs your jobs. The job results are displayed in pipeline. Pipeline arrangement begins with jobs.

For example:

Build-job1:
  script: "execute-build-job1"

test-job2:
  script: "execute-test-job2"

deploy-job3:
  script: “execute-deploy-job3”

Above example is the easiest gitlab CI/CD configuration with three different jobs, where each of the jobs executes a separate command.

Runners are fetched all above jobs and executed in the environment of the runner.

View jobs in a GitLab CI pipeline

Go to project -> build -> pipeline. You can see result of your jobs in pipeline.  By Selecting an individual job shows you its job log, and allows you to:

• Cancel the Job: If the job is currently running, you can cancel it to stop its execution.
• Retry the Job: If the job has failed, you have the option to retry it, attempting to run it again.
• Run the Job Again: If the job has passed, you can choose to run it again.
• Erase the Job Log: If you want to remove the log output of the job, you can erase the job log.

Please note that the available options for each job may vary based on its current status and the GitLab project’s settings and permissions.

By navigating through the pipelines and individual jobs, you can get a comprehensive view of your CI/CD process, check job statuses, and take necessary actions based on the results.

Jobs status in GitLab CI

In GitLab CI/CD, jobs can have various statuses that provide information about their current state and progress. Here are the different job statuses along with their meanings:

• Failed: The job execution encountered an error or failure, and it did not complete successfully.
• Warning: Some issues or warnings were encountered during the job execution, but it did not result in a complete failure.
• Pending: The job is waiting to be picked up and executed by a runner. This status occurs before the job starts running.
• Running: The job is currently being executed by a runner.
• Manual: The job is set to be manually triggered, and it will not run automatically as part of the regular pipeline flow. A user needs to manually trigger it.
• Scheduled: The job is scheduled to run at a specific time, and it will start executing at the scheduled time.
• Canceled: The job execution was canceled before it completed.
• Success: The job execution completed successfully without encountering any errors.
• Skipped: The job was skipped and did not run. This can happen when a job is configured to run conditionally and the conditions are not met.
• Created: The job was created but has not started running yet.

These statuses are crucial for understanding the state of each job in the CI/CD pipeline. They help users and developers identify which jobs succeeded, which failed, and which are pending or running, providing valuable insights for debugging and monitoring the pipeline’s health and stability.

Don’t use keyword as job names in GitLab CI:

In GitLab CI/CD configuration, job names must be unique, and certain keywords are reserved, meaning they cannot be used as job names. Attempting to use these keywords as job names will result in errors or unexpected behavior in your CI/CD pipeline configuration. Here are the reserved keywords that cannot be used as job names:

• image: Specifies the Docker image to use as the execution environment for the job.
• services: Defines additional services (such as databases) that should be linked to the job’s Docker container.
• stages: Defines the ordered list of stages that determine the sequence in which jobs are executed.
• types: Specifies the type of the job (e.g., test, deploy, build, etc.).
• before_script: Contains a list of shell commands that are run before each job.
• after_script: Contains a list of shell commands that are run after each job.
• variables: Defines variables that can be used within the job’s configuration.
• cache: Specifies files or directories that should be cached between job runs.
• include: Includes external CI/CD configuration files into the current pipeline.
• true: Boolean value indicating a true condition.
• false: Boolean value indicating a false condition.
• nil: Represents a null value or absence of a value.

To avoid conflicts and ensure a smooth CI/CD pipeline configuration, refrain from using these keywords as job names. Instead, use meaningful and descriptive names for your jobs that do not clash with reserved keywords. This will help maintain clarity and consistency in your CI/CD configuration files.

Group jobs in GitLab CI pipeline:

Grouping jobs in a CI/CD pipeline is a helpful way to organize related jobs together and display them as a single unit in the pipeline graph. As you mentioned, there are multiple symbols you can use to create job groups: number slash (/), colon (:), or space ( ). Here’s an example of how to group jobs using the number slash (/) symbol in the GitLab CI/CD pipeline configuration:

build java 1/3:
  stage: build
  script:
    - echo "java1"

build java 2/3:
  stage: build
  script:
    - echo "java2"

build java 3/3:
  stage: build
  script:
    - echo "java3"

In this example, we have three jobs named “build java 1/3,” “build java 2/3,” and “build java 3/3.” These jobs are part of a group named “build java,” which is indicated by the common prefix “build java” and the numbers 1/3, 2/3, and 3/3. The pipeline graph will display the “build java” group with three jobs, and each job shown with its corresponding number (1/3, 2/3, or 3/3).

It’s important to note that the symbols used to create job groups are interchangeable, so you can use either number slash (/), colon (:), or space ( ) to define job groups.

Hide Jobs in GitLab CI:

You can use the techniques mentioned to hide jobs in a GitLab CI/CD configuration file:

1. Commenting Out the Job’s Configuration: You can comment out the job’s configuration by adding the ‘#’ symbol before each line of the job definition. For example:

# hidden_job:
#   script:
#     - run test

By commenting out the job configuration, GitLab CI/CD will ignore this job during the pipeline execution, effectively hiding it from being processed.

• Using Job Names Starting with a Dot (.): Jobs whose names start with a dot (.) will not be processed by GitLab CI/CD. For instance:

.hidden_job:
  script:
    - run test

In this case, the job named “hidden_job” will be hidden, and GitLab CI/CD will not include it in the pipeline execution.

By using above techniques, you can hide jobs, prevent them from executing of your GitLab CI/CD pipelines.

Parameters used in GitLab CI Jobs:

In a GitLab CI/CD pipeline configuration, jobs are defined using a set of parameters that determine their behavior during execution. The parameters used in defining jobs:

• script: (Required) Defines a shell script that is executed by the Runner when the job is run.
• image: (Optional) Specifies the Docker image to use as the execution environment for the job.
• services: (Optional) Defines additional Docker services (e.g., databases) that should be linked to the job’s Docker container.
• stage: (Optional) Defines the job’s stage in the pipeline. The default value is “test” if not specified.
• type: (Optional) An alias for the “stage” parameter, used to define the job’s stage.
• variables: (Optional) Defines job-specific variables that can be used during job execution.
• only: (Optional) Defines a list of Git refs for which the job is created. The job will only run when the pipeline is triggered by these refs.
• except: (Optional) Defines a list of Git refs for which the job is not created. The job will not run when the pipeline is triggered by these refs.
• tags: (Optional) Defines a list of tags that are used to select a specific Runner to execute the job.
• allow_failure: (Optional) Allows the job to fail without affecting the commit status. Failed jobs with this parameter set will not mark the pipeline as failed.
• when: (Optional) Defines when the job should run. It can be set to “on_success,” “on_failure,” “always,” or “manual.”
• dependencies: (Optional) Defines other jobs that the current job depends on. This allows passing artifacts between jobs.
• artifacts: (Optional) Defines a list of job artifacts that should be saved for future use or for downstream jobs.
• cache: (Optional) Defines a list of files that should be cached between subsequent runs to improve performance.
• before_script: (Optional) Overrides a set of commands that are executed before the main script of the job.
• after_script: (Optional) Overrides a set of commands that are executed after the main script of the job.
• environment: (Optional) Defines the name of the environment to deploy to when the job is used for deployment purposes.
• coverage: (Optional) Defines code coverage settings for a given job to measure test coverage.
• retry: (Optional) Defines how many times a job can be auto-retried in case of failure.

These parameters provide a high level of customization and control over job execution in GitLab CI/CD pipelines, allowing users to tailor each job’s behavior to their specific requirements and integration needs.

Stages in GitLab CI:

The stages keyword is used to define the stages used by jobs in a CI/CD pipeline. The order of elements in stages defines the ordering of job execution.

The main points to note are:

1. Jobs of the same stage are executed in parallel, which means they can run simultaneously, taking advantage of multi-runner environments.
2. Jobs in the next stage are run only after all the jobs in the previous stage complete successfully. This ensures that the pipeline progresses in a sequential manner, and later stages depend on the successful completion of earlier stages.

For Example:

stages:
  - build
  - test
  - deploy

job1:
  stage: build
  script:
    - echo "Building..."

job2:
  stage: build
  script:
    - echo "Building something else..."

job3:
  stage: test
  script:
    - echo "Running tests..."

job4:
  stage: deploy
  script:
    - echo "Deploying..."

Execution Flow:

1. First, all jobs in the build stage (job1 and job2) are executed in parallel.
2. If both jobs in the build stage succeed, the job in the test stage (job3) is executed.
3. If the job in the test stage (job3) succeeds, the job in the deploy stage (job4) is executed.
4. If all jobs in the deploy stage succeed, the commit is marked as passed.
5. If any of the previous jobs fail (e.g., job1, job2, or job3), the commit is marked as failed, and no jobs from the later stages (test and deploy) are executed.

Edge Cases:

1. If no stages are defined in .gitlab-ci.yml, then the default stages “build,” “test,” and “deploy” are implicitly used.
2. If a job doesn’t specify a stage, it is automatically assigned to the test stage by default.

By using the stages keyword, you can structure your CI/CD pipeline in a clear and organized manner, ensuring jobs are executed appropriately and efficiently.

Script parameter in GitLab CI:

The script parameter is used to define the shell commands that will be executed by the Runner when running a specific job.

The script parameter can be specified as a single string or as an array of strings. In the array format, each element represents a separate command to be executed in sequence. Here are examples of both formats:

1. Single String Format:

job:
  script: "bundle exec rspec"

In this example, the job will execute the shell command “bundle exec rspec” using the Runner.

2. Array Format:

job:
  script:
    - uname -a
    - bundle exec rspec

In this example, the job will execute two commands sequentially: “uname -a” followed by “bundle exec rspec.”

Wrapping Script Commands in Quotes in GitLab CI:

As you mentioned, sometimes it’s necessary to wrap script commands in single (‘ ‘) or double (” “) quotes, especially when the command contains special characters such as colons (:) or when the command includes multiple words.

For example, if you want to execute a command that contains a colon, you need to wrap the command in quotes to avoid confusion with YAML syntax. Here’s an example:

job:
  script: "echo 'Hello: World'"

In this example, the command “echo ‘Hello: World’” is wrapped in single quotes (‘ ‘), so the YAML parser treats it as a single string.

Caution with Special Characters in GitLab CI:

It’s essential to be careful when using special characters within script commands. Characters like :, {, }, [, ], ,, &, *, #, ?, |, -, <, >, =, !, %, @, and ` might have special meanings in YAML or the shell, so they need to be handled properly. If a command includes any of these special characters, you should consider wrapping the command in quotes.

In this article we have covered what is Jobs in GitLab CI | GitLab CI Jobs and Stages in GitLab CI ?.

Related Articles:

Run your first GitLab CI CD pipeline [2 Steps]

Reference:

Jobs in GitLab CI official page