New feature

We are very excited to introduce the Ably Kafka Connector. It provides a ready-made integration between Kafka and Ably, allowing for realtime event distribution from Kafka to web, mobile, and IoT clients, over Ably’s feature-rich, multi-protocol pub/sub channels. This connector is verified by Confluent as Gold standard, following the guidelines set forth by Confluent’s Verified Integrations Program.

The Ably Kafka Connector makes it easy and seamless to use Ably as an internet-facing extension of Kafka. Both Ably and Kafka are event-driven solutions, sharing similar guarantees, messaging semantics, and characteristics. Kafka was built to work within a private network, and is not designed for distributing events between internal systems and consumers over the public internet. Ably was designed for edge streaming, and can act as the Internet-facing messaging layer to sit between Kafka and client devices when you’re extending Kafka to the edge.

How does the Ably Kafka Connector work?

The Ably Kafka Connector is a sink connector built on top of Kafka Connect. It can be self-hosted or hosted with a third-party provider — the most common being the Confluent Platform. You can download it from either GitHub or Confluent Cloud and install it into your Kafka Connect workers.

The Ably Kafka Connector provides a ready-made integration between Kafka and Ably via API Key. Once installed, you can configure the connector with your Ably API key to enable data from one or more Kafka topics to be published into a single Ably channel. Events are then distributed in realtime to web, mobile, and IoT clients over feature-rich, multi-protocol pub/sub Ably channels optimized for last-mile delivery.

To see Ably and Kafka working together connected by the Ably Kafka Connector, you can follow this step-by-step technical tutorial for building a realtime ticket booking solution with Kafka, FastAPI, and Ably.

How do I get started?

The Ably Kafka Connector is a gold standard connector on the Confluent Platform and can be installed from there if you’re deploying Kafka with Confluent. It can also be deployed locally using Docker. For more information on deployment and configuration, check out the documentation.

How do I give feedback?

The Ably Kafka Connector is available under the Apache 2 open source license and we are planning to continue extending and improving it, so we encourage feedback and feature requests; please either raise issues or pull requests if you would like to talk about contributing or feature requests. You can also contact us at any time.

Improvement

API keys are used to authenticate with the Ably platform's realtime and REST endpoints. A component of an API key is the key secret, which can be used to sign Ably JWT tokens using the HS256 algorithm. Per the specification, this value must be at least 32 bytes; Ably now produces 32-byte key secrets by default for compatibility with this specification.

This support for longer key secrets does not affect the operation of existing API keys. However, if you have opted to use Ably JWT tokens, you may consider cycling the relevant keys for additional security. Please refer to our API keys documentation for more information.

If you are storing an API key or key secret in fixed-width storage, you should check the maximum size of this field before using a 32-byte key secret, as the value will be longer.

This update is rolling out progressively over the week commencing 4th October, 2021.

How do I give feedback on this feature?

We rely on your feedback to improve this feature; please either raise issues or pull requests. You can also contact us at any time if you would like to talk about contributing or feature requests.

Client Library SDK Fix

Fixes bug #715, affecting those supporting push notifications - users cannot reactivate device after deactivating.

Client Library SDK Fix

Fixed bugs

• Add TypeScript support for REST publish parameters #785
• Fix a bug with parsing of authUrl responses #793

New feature

Using inband channel occupancy events, Ably users can subscribe to occupancy metadata events relating to a single channel. These events are then delivered to the client as messages on the channel itself. 

This feature allows developers to understand the exact numbers of connections, publishers, subscribers, presence connections, presence members, and presence subscribers relating to a channel. This information can be used for debugging and monitoring purposes, or for business logic - such as publishing a certain announcement on the channel only when you reach a certain number of subscribers.  

How does it work?

Inband channel occupancy events are added in addition to existing occupancy metachannel and Channel Status API, and provides a convenient interface to work with Occupancy in a streaming manner and within the scope of a single channel. Read more about inband channel occupancy in our documentation.

How do I get started with Inband channel occupancy events?

Take a look at this step-by-step tutorial to learn how to subscribe to inband channel occupancy events.

How do I give feedback on this feature?

We rely on your feedback to improve this feature; please either raise issues or pull requests. You can also contact us at any time if you would like to talk about contributing or feature requests.

Client Library SDK Fix

Fixed bugs:

• Specify channel modes to limit the amount of unnecessary data being sent and received over Ably #412
• EnhancedLocationUpdate timestamps do not represent real world time #387

Early access

Client Library SDK Improvement

Ably Flutter v1.2.2 is now available for preview, adding support for push notifications, including device activation with Ably (including device registration with APNs / FCM) and subscribing devices for push notifications. Check out the dedicated documentation and the example app for more information.

Full Changelog

Implemented enhancements:

• Push Notifications Device Registration (activation) and device subscription #107

Merged pull requests:

• Add documentation for token authentication #155 (ben-xD)
• Push notifications (activation and subscription) #140 (ben-xD)

Client Library SDK Fix

Implemented enhancements:

• Update Stats fields with latest MessageTraffic types #394
• Replace ULID with Android's UUID #680

Fixed bugs:

• Push Activation State Machine exception handling needs improvement #685
• WebsocketNotConnectedException on send #430

Client Library SDK Fix

Fixed bugs

• Implement RTL5b and RTL5j cases for detaching from channels in suspended/failed states #784

Client Library SDK Improvement

Implemented enhancements

• Add .once method to EventEmitter #779
• Implement Ably-Agent connection param for ably-js and NodeJS versions #740

Fixed bugs

• Fix channel names as object prototype keys #777
• Bump ws from 5.2.2 to 5.2.3 #781