Troubleshooting CI/CD Test Failures in GitHub Actions YAML for Couchbase Flask Starter

There are many templates for building web applications nowadays. Couchbase Flask Starter is one of them. This is an open-source project, so I have the opportunity to contribute to it. Jackson Drew and I have worked on the CI/CD failed running test issue. Under our mentor Angel Velasquez's guidance, we learned how to approach the problem and opened a pull request for review.

About Couchbase Flask Starter

The Couchbase Flask Starter project helps developers build a backend application using Flask as the framework and Couchbase as the database. It provides configuration files, environment setup, unit tests, and instructions for setting up the backend and connecting to the Couchbase NoSQL cloud database.

However, there are challenges when using a project template. Starting a program from scratch helps developers fully understand the code from the beginning and they freely design the system program. When programmers work on the existing code, they have to spend time understanding the codebase, its logic, dependencies, etc. There are legacy codes that need to be maintained and when adding new features, developers must ensure compatibility with the existing code. Therefore, it is challenging to improve and update the code.

Challenge 1: Local Setup

The first challenge my team and I encountered was setting up the project on our local machines. The error message was:

ERROR: Failed building wheel for couchbase

After researching, my teammate Jackson found a solution. Because we used the latest Python version, it is incompatible with the Python version that Couchbase uses. Therefore, we downgraded the Python version to 3.8. Then, we attempted to run the application on a local machine after setting up the Couchbase environment variables.

The application is successfully executed

Challenge 2: Replicating the Issue

The next step is replicating the issue. While I was setting up the Couchbase environment variables, I noticed that changing the COUCHBASE_DEFAULT_BUCKET value to something different from the bucket name I originally set on Couchbase Capella allowed me to reproduce the issue I was assigned.

This project has a set of tests to ensure parts of a backend application work properly with Couchbase as a database. The tests connect to a Couchbase database using a username and password and a specific Couchbase bucket.

How Environment Variables Work

The app first looks for the environment variables for Couchbase in the .env file. If it cannot find them, it will use the default values in config.py. By default, COUCHBASE_CONNECTION_STRING is 'couchbase://localhost', so it will attempt to connect to a Couchbase instance locally which may not exist if Couchbase is not installed and configured on the local machine.

Therefore, the tests fail if environment variables are not set up correctly. My task is to fix these failing tests in test_app.py which are not finding the Couchbase bucket.

I successfully replicated the issue

I checked both .env and config.py files, and the tests are failing because no database exists.

Finding the Solution

Are there other ways to set the default environment variables besides using config.py or .env? How does the program connect to a default Couchbase database in this setup?

Typically, an application should have separate databases for testing and development. A test database lets the program run tests in an isolated environment without affecting the real data.

Considered Approaches

Mocking approach: We could create a mock connection to the Couchbase server, so the tests would think they connected to a real database. However, this approach is complex for this template.

Chosen approach: Instead of faking a connection just to pass the tests, why not skip them if the environment variables are not set?

My team proposed modifying the test functions to skip them if the environment variables required for Couchbase configuration were missing. By adding a condition to check for the existence of the COUCHBASE_CONNECTION_STRING, COUCHBASE_USERNAME, COUCHBASE_PASSWORD, and COUCHBASE_DEFAULT_BUCKET environment variables, the tests only execute when these variables are correctly set up.

Implementation

First, we created a helper function is_env_vars_not_configured() to return True if any Couchbase environment variables are undefined:

def is_env_vars_not_configured():
    return (
        os.getenv('COUCHBASE_CONNECTION_STRING') is None or
        os.getenv('COUCHBASE_USERNAME') is None or
        os.getenv('COUCHBASE_PASSWORD') is None or
        os.getenv('COUCHBASE_DEFAULT_BUCKET') is None
    )

Then, pytest.mark.skipif() was used to conditionally skip tests based on this function:

@pytest.mark.skipif(
    is_env_vars_not_configured(),
    reason="Skipping as environment variables are not configured"
)
def test_index(client):
    pass

@pytest.mark.skipif(
    is_env_vars_not_configured(),
    reason="Skipping as environment variables are not configured"
)
def test_create_and_get_document(client, couchbase_setup):
    pass

This resolved the CI/CD running tests failure issue.

Verification

To verify the solution, we ran the tests locally with and without the environment variables. The tests are skipped as intended when the variables are not set.

Conclusion

Finally, we opened a pull request for this issue. This solution allows the CI/CD pipeline to run successfully without requiring a full Couchbase setup in every environment, while still running the tests when the proper configuration is available.

Key Takeaways

Understanding legacy codebases takes time and patience
Python version compatibility is crucial for dependencies
Conditional test skipping is a pragmatic solution for environment-dependent tests
Proper environment variable management is essential for CI/CD pipelines

This article documents the collaborative effort of fixing CI/CD test failures in the Couchbase Flask Starter project.