Package | Description |
---|---|
com.hazelcast.jet.avro |
Apache Avro file read/write support for Hazelcast Jet.
|
com.hazelcast.jet.hadoop |
Apache Hadoop HDFS read/write support for Hazelcast Jet.
|
com.hazelcast.jet.pipeline |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
Modifier and Type | Method and Description |
---|---|
BatchSource<D> |
AvroSourceBuilder.build()
Convenience for
AvroSourceBuilder.build(BiFunctionEx) . |
<T> BatchSource<T> |
AvroSourceBuilder.build(BiFunctionEx<String,? super D,T> mapOutputFn)
Builds a custom Avro file
BatchSource with supplied components
and the output function mapOutputFn . |
static <D> BatchSource<D> |
AvroSources.files(String directory,
BiFunctionEx<String,org.apache.avro.generic.GenericRecord,D> mapOutputFn)
Convenience for
AvroSources.filesBuilder(String, SupplierEx) which
reads all the files in the supplied directory as generic records and
emits the results of transforming each generic record with the supplied
mapping function. |
static <D> BatchSource<D> |
AvroSources.files(String directory,
Class<D> datumClass)
Convenience for
AvroSources.filesBuilder(String, Class) which
reads all the files in the supplied directory as specific records using
supplied datumClass . |
Modifier and Type | Method and Description |
---|---|
static <K,V> BatchSource<Map.Entry<K,V>> |
HdfsSources.hdfs(org.apache.hadoop.mapred.JobConf jobConf)
Convenience for
HdfsSources.hdfs(JobConf, BiFunctionEx)
with Map.Entry as its output type. |
static <K,V,E> BatchSource<E> |
HdfsSources.hdfs(org.apache.hadoop.mapred.JobConf jobConf,
BiFunctionEx<K,V,E> projectionFn)
Returns a source that reads records from Apache Hadoop HDFS and emits
the results of transforming each record (a key-value pair) with the
supplied mapping function.
|
Modifier and Type | Method and Description |
---|---|
static <T> BatchSource<T> |
Sources.batchFromProcessor(String sourceName,
ProcessorMetaSupplier metaSupplier)
Returns a bounded (batch) source constructed directly from the given
Core API processor meta-supplier.
|
BatchSource<T> |
SourceBuilder.Batch.build()
Builds and returns the batch source.
|
BatchSource<String> |
FileSourceBuilder.build()
Convenience for
FileSourceBuilder.build(BiFunctionEx) . |
<T> BatchSource<T> |
FileSourceBuilder.build(BiFunctionEx<String,String,? extends T> mapOutputFn)
Builds a custom file
BatchSource with supplied components and the
output function mapOutputFn . |
static <K,V> BatchSource<Map.Entry<K,V>> |
Sources.cache(String cacheName)
Returns a source that fetches entries from a Hazelcast
ICache
with the given name and emits them as Map.Entry . |
static BatchSource<String> |
Sources.files(String directory)
A source to read all files in a directory in a batch way.
|
static <T> BatchSource<T> |
Sources.jdbc(String connectionURL,
String query,
FunctionEx<? super ResultSet,? extends T> createOutputFn)
Convenience for
Sources.jdbc(SupplierEx,
ToResultSetFunction, FunctionEx) . |
static <T> BatchSource<T> |
Sources.jdbc(SupplierEx<? extends Connection> connectionSupplier,
ToResultSetFunction resultSetFn,
FunctionEx<? super ResultSet,? extends T> createOutputFn)
Returns a source which connects to the specified database using the given
connectionSupplier , queries the database and creates a result set
using the the given resultSetFn . |
static <T> BatchSource<T> |
Sources.list(IList<? extends T> list)
Returns a source that emits items retrieved from a Hazelcast
IList . |
static <T> BatchSource<T> |
Sources.list(String listName)
Returns a source that emits items retrieved from a Hazelcast
IList . |
static <K,V> BatchSource<Map.Entry<K,V>> |
Sources.map(IMap<? extends K,? extends V> map)
Returns a source that fetches entries from the given Hazelcast
IMap and emits them as Map.Entry . |
static <T,K,V> BatchSource<T> |
Sources.map(IMap<? extends K,? extends V> map,
Predicate<? super K,? super V> predicate,
FunctionEx<? super Map.Entry<K,V>,? extends T> projectionFn)
Convenience for
Sources.map(IMap, Predicate, Projection) which uses a
FunctionEx as the projection function. |
static <T,K,V> BatchSource<T> |
Sources.map(IMap<? extends K,? extends V> map,
Predicate<? super K,? super V> predicate,
Projection<? super Map.Entry<K,V>,? extends T> projection)
Returns a source that fetches entries from the given Hazelcast
IMap . |
static <K,V> BatchSource<Map.Entry<K,V>> |
Sources.map(String mapName)
Returns a source that fetches entries from a local Hazelcast
IMap
with the specified name and emits them as Map.Entry . |
static <T,K,V> BatchSource<T> |
Sources.map(String mapName,
Predicate<? super K,? super V> predicate,
FunctionEx<? super Map.Entry<K,V>,? extends T> projectionFn)
Convenience for
Sources.map(String, Predicate, Projection)
which uses a FunctionEx as the projection function. |
static <T,K,V> BatchSource<T> |
Sources.map(String mapName,
Predicate<? super K,? super V> predicate,
Projection<? super Map.Entry<K,V>,? extends T> projection)
Returns a source that fetches entries from a local Hazelcast
IMap with the specified name. |
static <K,V> BatchSource<Map.Entry<K,V>> |
Sources.remoteCache(String cacheName,
ClientConfig clientConfig)
Returns a source that fetches entries from the Hazelcast
ICache
with the specified name in a remote cluster identified by the supplied
ClientConfig and emits them as Map.Entry . |
static <T> BatchSource<T> |
Sources.remoteList(String listName,
ClientConfig clientConfig)
Returns a source that emits items retrieved from a Hazelcast
IList in a remote cluster identified by the supplied ClientConfig . |
static <K,V> BatchSource<Map.Entry<K,V>> |
Sources.remoteMap(String mapName,
ClientConfig clientConfig)
Returns a source that fetches entries from the Hazelcast
IMap
with the specified name in a remote cluster identified by the supplied
ClientConfig and emits them as Map.Entry . |
static <T,K,V> BatchSource<T> |
Sources.remoteMap(String mapName,
ClientConfig clientConfig,
Predicate<? super K,? super V> predicate,
FunctionEx<? super Map.Entry<K,V>,? extends T> projectionFn)
Convenience for
Sources.remoteMap(String, ClientConfig, Predicate, Projection)
which use a FunctionEx as the projection function. |
static <T,K,V> BatchSource<T> |
Sources.remoteMap(String mapName,
ClientConfig clientConfig,
Predicate<? super K,? super V> predicate,
Projection<? super Map.Entry<K,V>,? extends T> projection)
Returns a source that fetches entries from a remote Hazelcast
IMap with the specified name in a remote cluster identified by the
supplied ClientConfig . |
Modifier and Type | Method and Description |
---|---|
<T> BatchStage<T> |
Pipeline.drawFrom(BatchSource<? extends T> source)
Returns a pipeline stage that represents a bounded (batch) data source.
|
Copyright © 2019 Hazelcast, Inc.. All rights reserved.