org.apache.samza.system

Interface SystemAdmin

All Known Implementing Classes:

SinglePartitionWithoutOffsetsSystemAdmin

public interface SystemAdmin

Helper interface attached to an underlying system to fetch information about streams, partitions, offsets, etc. This interface is useful for providing utility methods that Samza needs in order to interact with a system.

Method Summary

Modifier and Type	Method and Description
default boolean	clearStream(StreamSpec streamSpec) Clear the entire stream described by the spec.
default boolean	createStream(StreamSpec streamSpec) Create a stream described by the spec.
default void	deleteMessages(java.util.Map<SystemStreamPartition,java.lang.String> offsets) Delete records up to (and including) the provided ssp offsets for all system stream partitions specified in the map
default java.util.Set<SystemStream>	getAllSystemStreams() Fetch the set of all available streams
java.util.Map<SystemStreamPartition,java.lang.String>	getOffsetsAfter(java.util.Map<SystemStreamPartition,java.lang.String> offsets) Fetches the offsets for the messages immediately after the supplied offsets for a group of SystemStreamPartitions.
default java.util.Map<SystemStreamPartition,SystemStreamMetadata.SystemStreamPartitionMetadata>	getSSPMetadata(java.util.Set<SystemStreamPartition> ssps) Fetch metadata from a system for a set of SSPs.
java.util.Map<java.lang.String,SystemStreamMetadata>	getSystemStreamMetadata(java.util.Set<java.lang.String> streamNames) Fetch metadata from a system for a set of streams.
default java.util.Map<java.lang.String,SystemStreamMetadata>	getSystemStreamPartitionCounts(java.util.Set<java.lang.String> streamNames, long cacheTTL) Get partitions counts only.
default boolean	isStopped() To give the status of current systemAdmin
java.lang.Integer	offsetComparator(java.lang.String offset1, java.lang.String offset2) Compare the two offsets.
default java.lang.String	resolveStartpointToOffset(SystemStreamPartition systemStreamPartition, Startpoint startpoint) Resolves the startpoint to a system specific offset.
default void	start()
default void	stop()
default void	validateStream(StreamSpec streamSpec) Validates the stream described by the streamSpec on the system.

Method Detail

start

default void start()

stop

default void stop()

isStopped

default boolean isStopped()

To give the status of current systemAdmin

Returns:

boolean stopped or not;

getOffsetsAfter

java.util.Map<SystemStreamPartition,java.lang.String> getOffsetsAfter(java.util.Map<SystemStreamPartition,java.lang.String> offsets)

Fetches the offsets for the messages immediately after the supplied offsets for a group of SystemStreamPartitions.

Parameters:

offsets - Map from SystemStreamPartition to current offsets.

Returns:

Map from SystemStreamPartition to offsets immediately after the current offsets.

getSystemStreamMetadata

java.util.Map<java.lang.String,SystemStreamMetadata> getSystemStreamMetadata(java.util.Set<java.lang.String> streamNames)

Fetch metadata from a system for a set of streams.

Parameters:

streamNames - The streams to to fetch metadata for.

Returns:

A map from stream name to SystemStreamMetadata for each stream requested in the parameter set.

getSSPMetadata

default java.util.Map<SystemStreamPartition,SystemStreamMetadata.SystemStreamPartitionMetadata> getSSPMetadata(java.util.Set<SystemStreamPartition> ssps)

Fetch metadata from a system for a set of SSPs. Implementors should override this if there is a more efficient implementation than delegating to getSystemStreamMetadata(java.util.Set<java.lang.String>).

Parameters:

ssps - SSPs for which to get metadata

Returns:

A map from SystemStreamPartition to the SystemStreamPartitionMetadata, with an entry for each SSP in ssps for which metadata could be found

Throws:

java.lang.RuntimeException - if there was an error fetching metadata

offsetComparator

java.lang.Integer offsetComparator(java.lang.String offset1,
                                   java.lang.String offset2)

Compare the two offsets. -1, 0, +1 means offset1 < offset2, offset1 == offset2 and offset1 > offset2 respectively. Return null if those two offsets are not comparable

Parameters:

offset1 - First offset for comparison.

offset2 - Second offset for comparison.

Returns:

-1 if offset1 < offset2; 0 if offset1 == offset2; 1 if offset1 > offset2. Null if not comparable

createStream

default boolean createStream(StreamSpec streamSpec)

Create a stream described by the spec.

Parameters:

streamSpec - The spec, or blueprint from which the physical stream will be created on the system.

Returns:

true if the stream was actually created and not pre-existing. false if the stream was pre-existing. A RuntimeException will be thrown if creation fails.

validateStream

default void validateStream(StreamSpec streamSpec)
                     throws StreamValidationException

Validates the stream described by the streamSpec on the system. A StreamValidationException should be thrown for any validation error.

Parameters:

streamSpec - The spec, or blueprint for the physical stream on the system.

Throws:

StreamValidationException - if validation fails.

clearStream

default boolean clearStream(StreamSpec streamSpec)

Clear the entire stream described by the spec.

Parameters:

streamSpec - The spec for the physical stream on the system.

Returns:

true if the stream was successfully cleared. false if clearing stream failed.

deleteMessages

default void deleteMessages(java.util.Map<SystemStreamPartition,java.lang.String> offsets)

Delete records up to (and including) the provided ssp offsets for all system stream partitions specified in the map

Parameters:

offsets - a map from system stream partition to offset

getSystemStreamPartitionCounts

default java.util.Map<java.lang.String,SystemStreamMetadata> getSystemStreamPartitionCounts(java.util.Set<java.lang.String> streamNames,
                                                                                            long cacheTTL)

Get partitions counts only. Should be more efficient then getSystemStreamMetadata, but if not implemented revert to getSystemStreamMetadata.

Parameters:

streamNames - set of streams to query.

cacheTTL - cacheTTL to use if caching the values.

Returns:

A map from stream name to SystemStreamMetadata for each stream requested in the parameter set.

getAllSystemStreams

default java.util.Set<SystemStream> getAllSystemStreams()

Fetch the set of all available streams

Returns:

The set of all available SystemStreams.

resolveStartpointToOffset

default java.lang.String resolveStartpointToOffset(SystemStreamPartition systemStreamPartition,
                                                   Startpoint startpoint)

Resolves the startpoint to a system specific offset.

Parameters:

startpoint - represents the startpoint.

systemStreamPartition - represents the system stream partition.

Returns:

the resolved offset.