Abstract—The poor scalability of Volunteer Computing (VC) hinders the application of it because a tremendous number of volunteers are needed in order to achieve the same performance as that of a traditional HPC. This paper explores optimization potential to improve the scalability of VC from three points of view. First, the heterogeneity of volunteers’ compute-capacity has been chosen from the whole spectrum of impact factors to study optimization potential. Second, a DTH (Distributed Hash Table) based supporting platform and MapReduce are fused together as the discussion context. Third, transformed versions of work stealing have been proposed to optimize VC for both compute- and data-intensive applications. On this basis, the proposed optimization strategies are evaluated by three steps. First, a proof-of-concept prototype is implemented to support the representation and testing of the proposed optimization strategies. Second, virtual tasks are composed to apply certain compute- or data-intensity on the running MapReduce. Third, the competence of VC, running the original equity strategy and the optimization strategies, is tested against the virtual tasks. The evaluation of results has confirmed that the impaired performance has been improved about 24.5% for compute-intensive applications and about 19.5% for data-intensive applications.

Index Terms—Optimization, volunteer computing, work stealing, big data, MapReduce, scalability.

Cite: Wei Li and William W. Guo, “The Optimization Potential of Volunteer Computing for Compute or Data Intensive Applications,” vol. 14, no. 10, pp. 971-979, 2019. Doi: 10.12720/jcm.14.10.971-979.