MapReduce

1) Java's ForkJoinPool

ForkJoinPool implements a thread pool following Java's Executor Service interface. It is designed to efficiently run a large number of tasks using a pool of worker threads that use work-stealing algorithm.

2) MapReduce Data Locality

The mapper is responsible for reading the input data and generating the intermediate key-value pairs. These intermediate key-value pairs contain the data locally, so not every task needs to read from the file. Further, the work-stealing algorithm in ForkJoinPool reduces the contention/caching issues observed in some spin lock implementations. In large distributed systems, where communication is more expensive and fault tolerance is a concern, this specific MapReduce framework may not be the best choice, but data-locality still helps in reducing the communication overhead.

3) MapReduce Lack of Synchronization Primitives

MapReduce Framework does not need synchronization primitives because the intermediate key-value pairs are stored locally, and each task uses these pairs to generate/reduce the final output.

4) MapReduce Limitations

MapReduce struggles to handle parallel problems that cannot be split into independent, communicationless chunks. In our use case, it was trivial that whatever input and intermediary data we needed to process was split between independent chunks from the files' content. This is an inherent issue to all performant data parallelism techniques, since data parallel tasks that require communication among tasks suffer from Amdal's law and are not trivial to implement without being extremely careful & clever.