If you ever developed against Cloud services, you probably know it’s a struggle to develop, debug or test locally without the access to the service itself. In the last years however, it is becoming more approachable due to the plenty of proprietary emulators being published by the cloud providers.

Recently, I’ve joined a project, where the main messaging system used for asynchronous communication was GCP Pub/Sub. Software, responsible for Pub/Sub connectivity is Spring Cloud Stream GCP with already stated as legacy - annotation-based approach. Conventions established was to disable Pub/Sub support for local development and to mock Sink and Source in the tests. That’s fine unless:

I’m not a big fan of mocks, especially when it comes to integration testing, where I usually treat them as a last resort. Saying that, below are the ingredients, which allowed me to work locally with GCP Pub Sub with ease.

GCP Pub/Sub Emulator

GCP Pub/Sub Emulator plays the key part in the solution. It emulates production (cloud) version of Pub / Sub services with certain limitations:

  • UpdateTopic and UpdateSnapshot RPCs are not supported.

  • IAM operations are not supported.

  • Configurable message retention is not supported; all messages are retained indefinitely.

  • Retry policies are not supported.

  • Subscription expiration is not supported. Subscriptions do not expire.

  • Filtering is not supported.

— Emulator reference

Retry policies not being supported being the most problematic, because in case of an exception we would get stuck in an infinite redelivery loop (for Pub/Sub subscriber). We’ll handle that later on.

All the basics (installation, running, etc.) are pretty well explained in the official documentation. I’d like to focus on the ease of use aspect.

Docker is your friend

I think it’s no surprise, that Docker is mentioned here. With Docker, you can abstract away all, less important details (dependencies, installation process) of the service you want to use and deploy it in no-time. Hence, it comes in very handy if we talk testing (with great Testcontainers library) or local development.

I have created Pub/Sub Emulator docker image which is available at the Docker Hub. If you’d like to build the image yourself, source code is available at Github. To run the Pub/Sub Emulator with sensible defaults run Docker command as follows: docker run -d -p 8085:8085 poznachowski/gcp-pubsub-emulator:latest
If more dependencies are required to set up the local environment, it makes sense to wrap them into a single docker-compose file:

docker-compose.yml
1
2
3
4
5
services:
  pubsub_emulator:
    image: poznachowski/gcp-pubsub-emulator:latest
    ports:
    - 8085:8085

One important variable is Project ID which default value is local.

Spring Boot / Cloud configuration

Now, we need to tell Srping to use the local version of the Pub/Sub. To do this, we will make use of an additional local profile with following configuration:

application-local.yml
spring:
  cloud:
    gcp:
      pubsub:
        enabled: true
        emulator-host: localhost:8085
        project-id: local (1)
    stream:
      gcp:
        pubsub:
          default:
            consumer:
              ack-mode: AUTO_ACK
              auto-create-resources: true
            producer:
              auto-create-resources: true
1 This is the only interesting part here.

{{% admonition tip %}} It’s possible that this configuration is split into application-local.yml and bootstrap-local.yml depending on the project structure / conventions. Please be aware that, with Spring Cloud 2020.0 Bootstrap is disabled by default. {{% /admonition %}}

1
2
3
4
5
6
7
8
@Documented
@Inherited
@Target({ElementType.TYPE, ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
@MockBean(value = Clock.class, reset = MockReset.NONE) (1)
public @interface FixedClock {
    String value() default "2010-01-10T10:00:00Z";
}