org.apache.samza.operators

Interface MessageStream<M extends MessageEnvelope>

Type Parameters:

M - type of MessageEnvelopes in this stream

@InterfaceStability.Unstable
public interface MessageStream<M extends MessageEnvelope>

Represents a stream of MessageEnvelopes.

A MessageStream can be transformed into another MessageStream by applying the transforms in this API.

Method Summary

Modifier and Type	Method and Description
MessageStream<M>	filter(FilterFunction<M> filterFn) Applies the provided FilterFunction to MessageEnvelopes in this MessageStream and returns the transformed MessageStream.
<TM extends MessageEnvelope> MessageStream<TM>	flatMap(FlatMapFunction<M,TM> flatMapFn) Applies the provided 1:n FlatMapFunction to transform a MessageEnvelope in this MessageStream to n MessageEnvelopes in the transformed MessageStream
<K,OM extends MessageEnvelope<K,?>,RM extends MessageEnvelope> MessageStream<RM>	join(MessageStream<OM> otherStream, JoinFunction<M,OM,RM> joinFn) Joins this MessageStream with another MessageStream using the provided pairwise JoinFunction.
<TM extends MessageEnvelope> MessageStream<TM>	map(MapFunction<M,TM> mapFn) Applies the provided 1:1 MapFunction to MessageEnvelopes in this MessageStream and returns the transformed MessageStream.
MessageStream<M>	merge(java.util.Collection<MessageStream<M>> otherStreams) Merge all otherStreams with this MessageStream.
void	sink(SinkFunction<M> sinkFn) Allows sending MessageEnvelopes in this MessageStream to an output SystemStream using the provided SinkFunction.
<K,WV,WM extends WindowPane<K,WV>> MessageStream<WM>	window(Window<M,K,WV,WM> window) Groups and processes the MessageEnvelopes in this MessageStream according to the provided Window (e.g.

Method Detail

map

<TM extends MessageEnvelope> MessageStream<TM> map(MapFunction<M,TM> mapFn)

Applies the provided 1:1 MapFunction to MessageEnvelopes in this MessageStream and returns the transformed MessageStream.

Type Parameters:

TM - the type of MessageEnvelopes in the transformed MessageStream

Parameters:

mapFn - the function to transform a MessageEnvelope to another MessageEnvelope

Returns:

the transformed MessageStream

flatMap

<TM extends MessageEnvelope> MessageStream<TM> flatMap(FlatMapFunction<M,TM> flatMapFn)

Applies the provided 1:n FlatMapFunction to transform a MessageEnvelope in this MessageStream to n MessageEnvelopes in the transformed MessageStream

Type Parameters:

TM - the type of MessageEnvelopes in the transformed MessageStream

Parameters:

flatMapFn - the function to transform a MessageEnvelope to zero or more MessageEnvelopes

Returns:

the transformed MessageStream

filter

MessageStream<M> filter(FilterFunction<M> filterFn)

Applies the provided FilterFunction to MessageEnvelopes in this MessageStream and returns the transformed MessageStream.

The FilterFunction is a predicate which determines whether a MessageEnvelope in this MessageStream should be retained in the transformed MessageStream.

Parameters:

filterFn - the predicate to filter MessageEnvelopes from this MessageStream

Returns:

the transformed MessageStream

sink

void sink(SinkFunction<M> sinkFn)

Allows sending MessageEnvelopes in this MessageStream to an output SystemStream using the provided SinkFunction.

Parameters:

sinkFn - the function to send MessageEnvelopes in this stream to output systems

window

<K,WV,WM extends WindowPane<K,WV>> MessageStream<WM> window(Window<M,K,WV,WM> window)

Groups and processes the MessageEnvelopes in this MessageStream according to the provided Window (e.g. tumbling, sliding or session windows) and returns the transformed MessageStream of WindowPanes.

Use the Windows helper methods to create the appropriate windows.

Type Parameters:

K - the type of key in the MessageEnvelope in this MessageStream. If a key is specified, panes are emitted per-key.

WV - the type of value in the WindowPane in the transformed MessageStream

WM - the type of WindowPane in the transformed MessageStream

Parameters:

window - the window to group and process MessageEnvelopes from this MessageStream

Returns:

the transformed MessageStream

join

<K,OM extends MessageEnvelope<K,?>,RM extends MessageEnvelope> MessageStream<RM> join(MessageStream<OM> otherStream,
                                                                                      JoinFunction<M,OM,RM> joinFn)

Joins this MessageStream with another MessageStream using the provided pairwise JoinFunction.

We currently only support 2-way joins.

Type Parameters:

K - the type of join key

OM - the type of MessageEnvelopes in the other stream

RM - the type of MessageEnvelopes resulting from the joinFn

Parameters:

otherStream - the other MessageStream to be joined with

joinFn - the function to join MessageEnvelopes from this and the other MessageStream

Returns:

the joined MessageStream

merge

MessageStream<M> merge(java.util.Collection<MessageStream<M>> otherStreams)

Merge all otherStreams with this MessageStream.

The merging streams must have the same MessageEnvelope type M.

Parameters:

otherStreams - other MessageStreams to be merged with this MessageStream

Returns:

the merged MessageStream