org.infinispan.distexec.mapreduce

Class MapReduceManagerImpl

java.lang.Object

org.infinispan.distexec.mapreduce.MapReduceManagerImpl

All Implemented Interfaces:

MapReduceManager

public class MapReduceManagerImpl
extends Object
implements MapReduceManager

Default implementation of MapReduceManager.

This is an internal class, not intended to be used by clients.

Since:

5.2

Author:

Vladimir Blagojevic

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MapReduceManagerImpl.DeltaAwareListExternalizer
static class	MapReduceManagerImpl.DeltaListExternalizer
static class	MapReduceManagerImpl.IntermediateKey<V> IntermediateCompositeKey

Method Summary

Methods

Modifier and Type	Method and Description
protected <KIn,VIn,KOut,VOut> void	combine(MapCombineCommand<KIn,VIn,KOut,VOut> mcc, org.infinispan.distexec.mapreduce.MapReduceManagerImpl.CollectableCollector<KOut,VOut> c)
protected <KIn> Set<KIn>	filterLocalPrimaryOwner(Set<KIn> nodeLocalKeys, DistributionManager dm)
ExecutorService	getExecutorService() ExecutorService provided for local task execution
void	init(EmbeddedCacheManager cacheManager, PersistenceManager persistenceManager, ExecutorService asyncTransportExecutor, ClusteringDependentLogic cdl, TimeService timeService, Configuration configuration)
protected <KIn,VIn,KOut,VOut> org.infinispan.distexec.mapreduce.MapReduceManagerImpl.CollectableCollector<KOut,VOut>	map(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)
protected <KIn,VIn,KOut,VOut> Set<KOut>	mapAndCombine(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)
<KIn,VIn,KOut,VOut> Set<KOut>	mapAndCombineForDistributedReduction(MapCombineCommand<KIn,VIn,KOut,VOut> mcc) Invoked when MapCombineCommand arrives to a target Infinispan node and returns set of intermediate keys that needs to be reduced.
<KIn,VIn,KOut,VOut> Map<KOut,List<VOut>>	mapAndCombineForLocalReduction(MapCombineCommand<KIn,VIn,KOut,VOut> mcc) Invoked when MapCombineCommand arrives to a target Infinispan node and returns map of resulting values that need to be reduced.
<T> Map<List<T>>	mapKeysToNodes(DistributionManager dm, String taskId, Collection<T> keysToMap) Maps Map/Reduce task intermediate or input keys to nodes on Infinispan cluster
<KOut,VOut> Map<KOut,VOut>	reduce(ReduceCommand<KOut,VOut> reduceCommand) Invoked when ReduceCommand arrives to a target Infinispan node.
protected <KOut,VOut> void	reduce(ReduceCommand<KOut,VOut> reduceCommand, Map<KOut,VOut> result)
<KOut,VOut> void	reduce(ReduceCommand<KOut,VOut> reduceCommand, String resultCache) Invoked when ReduceCommand arrives to a target Infinispan node.

Method Detail

init

public void init(EmbeddedCacheManager cacheManager,
        PersistenceManager persistenceManager,
        ExecutorService asyncTransportExecutor,
        ClusteringDependentLogic cdl,
        TimeService timeService,
        Configuration configuration)

getExecutorService

public ExecutorService getExecutorService()

Description copied from interface: MapReduceManager

ExecutorService provided for local task execution

Specified by:

getExecutorService in interface MapReduceManager

Returns:

ExecutorService for local tasks

mapAndCombineForLocalReduction

public <KIn,VIn,KOut,VOut> Map<KOut,List<VOut>> mapAndCombineForLocalReduction(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)

Description copied from interface: MapReduceManager

Invoked when MapCombineCommand arrives to a target Infinispan node and returns map of resulting values that need to be reduced.

Unlike MapReduceManager.mapAndCombineForDistributedReduction(MapCombineCommand) implementations should return a map Map> which should be ready for reduction at master Infinispan node

Specified by:

mapAndCombineForLocalReduction in interface MapReduceManager

Parameters:

mcc - MapCombineCommand sent from MapReduceTask

Returns:

a map Map> which should be ready for reduction at master Infinispan node

mapAndCombineForDistributedReduction

public <KIn,VIn,KOut,VOut> Set<KOut> mapAndCombineForDistributedReduction(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)

Description copied from interface: MapReduceManager

Invoked when MapCombineCommand arrives to a target Infinispan node and returns set of intermediate keys that needs to be reduced.

Specified by:

mapAndCombineForDistributedReduction in interface MapReduceManager

Parameters:

mcc - MapCombineCommand sent from MapReduceTask

Returns:

a set of intermediate keys to be reduced distributively

reduce

public <KOut,VOut> Map<KOut,VOut> reduce(ReduceCommand<KOut,VOut> reduceCommand)

Description copied from interface: MapReduceManager

Invoked when ReduceCommand arrives to a target Infinispan node. Implementations should return a map of reduced output keys and values to be returned to invoker of MapReduceTask

Specified by:

reduce in interface MapReduceManager

Parameters:

reduceCommand - ReduceCommand sent from MapReduceTask

Returns:

map of reduced output keys and values returned to MapReduceTask

reduce

public <KOut,VOut> void reduce(ReduceCommand<KOut,VOut> reduceCommand,
                      String resultCache)

Description copied from interface: MapReduceManager

Invoked when ReduceCommand arrives to a target Infinispan node. However, instead of returning a resulting Map to master node, reduce command emits results of reduce phase to a specified resulting cache.

Specified by:

reduce in interface MapReduceManager

Parameters:

reduceCommand - ReduceCommand sent from MapReduceTask

resultCache - result cache to store results of reduce phase

reduce

protected <KOut,VOut> void reduce(ReduceCommand<KOut,VOut> reduceCommand,
                      Map<KOut,VOut> result)

map

protected <KIn,VIn,KOut,VOut> org.infinispan.distexec.mapreduce.MapReduceManagerImpl.CollectableCollector<KOut,VOut> map(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)

mapAndCombine

protected <KIn,VIn,KOut,VOut> Set<KOut> mapAndCombine(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)

combine

protected <KIn,VIn,KOut,VOut> void combine(MapCombineCommand<KIn,VIn,KOut,VOut> mcc,
                               org.infinispan.distexec.mapreduce.MapReduceManagerImpl.CollectableCollector<KOut,VOut> c)

mapKeysToNodes

public <T> Map<List<T>> mapKeysToNodes(DistributionManager dm,
                              String taskId,
                              Collection<T> keysToMap)

Description copied from interface: MapReduceManager

Maps Map/Reduce task intermediate or input keys to nodes on Infinispan cluster

Specified by:

mapKeysToNodes in interface MapReduceManager

Parameters:

dm - distribution manager to use for locating keys on hash wheel

taskId - id of the map/reduce task

keysToMap - list of input keys to locate in the cluster

Returns:

map where each key is an Address in the cluster and value are the keys mapped to that Address

filterLocalPrimaryOwner

protected <KIn> Set<KIn> filterLocalPrimaryOwner(Set<KIn> nodeLocalKeys,
                                     DistributionManager dm)