Apache Hadoop This is a benchmark of the Apache Hadoop making use of its built-in name-node throughput benchmark (NNThroughputBenchmark).
To run this test with the Phoronix Test Suite , the basic command is: phoronix-test-suite benchmark hadoop .
Test Created 7 September 2023
Test Type System
Average Install Time 12 Seconds
Average Run Time 12 Minutes, 5 Seconds
Test Dependencies Java
Accolades 5k+ Downloads Public Result Uploads * Reported Installs ** Reported Test Completions ** Test Profile Page Views OpenBenchmarking.org Events Apache Hadoop Popularity Statistics pts/hadoop 2023.09 2023.10 2023.11 2023.12 2024.01 2024.02 2024.03 2024.04 700 1400 2100 2800 3500
* Uploading of benchmark result data to OpenBenchmarking.org is always optional (opt-in) via the Phoronix Test Suite for users wishing to share their results publicly. ** Data based on those opting to upload their test results to OpenBenchmarking.org and users enabling the opt-in anonymous statistics reporting while running benchmarks from an Internet-connected platform. Data updated weekly as of 29 April 2024.
Rename 19.2% Open 22.0% Delete 20.6% File Status 14.4% Create 23.9% Operation Option Popularity OpenBenchmarking.org
500 10.4% 20 29.5% 50 33.3% 100 26.8% Threads Option Popularity OpenBenchmarking.org
1000000 38.5% 10000000 7.5% 100000 53.9% Files Option Popularity OpenBenchmarking.org
Revision Historypts/hadoop-1.0.0 [View Source ] Thu, 07 Sep 2023 14:37:09 GMT Initial commit of Hadoop benchmark.
Performance MetricsAnalyze Test Configuration: pts/hadoop-1.0.x - Operation: Create - Threads: 100 - Files: 100000 pts/hadoop-1.0.x - Operation: Create - Threads: 20 - Files: 100000 pts/hadoop-1.0.x - Operation: Create - Threads: 50 - Files: 100000 pts/hadoop-1.0.x - Operation: Delete - Threads: 50 - Files: 100000 pts/hadoop-1.0.x - Operation: Open - Threads: 50 - Files: 100000 pts/hadoop-1.0.x - Operation: Delete - Threads: 20 - Files: 100000 pts/hadoop-1.0.x - Operation: Rename - Threads: 50 - Files: 100000 pts/hadoop-1.0.x - Operation: Rename - Threads: 20 - Files: 100000 pts/hadoop-1.0.x - Operation: Create - Threads: 50 - Files: 1000000 pts/hadoop-1.0.x - Operation: Open - Threads: 20 - Files: 100000 pts/hadoop-1.0.x - Operation: Open - Threads: 50 - Files: 1000000 pts/hadoop-1.0.x - Operation: Delete - Threads: 50 - Files: 1000000 pts/hadoop-1.0.x - Operation: Open - Threads: 100 - Files: 100000 pts/hadoop-1.0.x - Operation: Rename - Threads: 50 - Files: 1000000 pts/hadoop-1.0.x - Operation: Open - Threads: 100 - Files: 1000000 pts/hadoop-1.0.x - Operation: Create - Threads: 100 - Files: 1000000 pts/hadoop-1.0.x - Operation: Rename - Threads: 100 - Files: 100000 pts/hadoop-1.0.x - Operation: Delete - Threads: 100 - Files: 100000 pts/hadoop-1.0.x - Operation: Open - Threads: 20 - Files: 1000000 pts/hadoop-1.0.x - Operation: File Status - Threads: 50 - Files: 100000 pts/hadoop-1.0.x - Operation: File Status - Threads: 20 - Files: 100000 pts/hadoop-1.0.x - Operation: Delete - Threads: 100 - Files: 1000000 pts/hadoop-1.0.x - Operation: Create - Threads: 20 - Files: 1000000 pts/hadoop-1.0.x - Operation: Delete - Threads: 20 - Files: 1000000 pts/hadoop-1.0.x - Operation: Rename - Threads: 20 - Files: 1000000 pts/hadoop-1.0.x - Operation: Create - Threads: 500 - Files: 100000 pts/hadoop-1.0.x - Operation: File Status - Threads: 50 - Files: 1000000 pts/hadoop-1.0.x - Operation: Rename - Threads: 100 - Files: 1000000 pts/hadoop-1.0.x - Operation: File Status - Threads: 100 - Files: 100000 pts/hadoop-1.0.x - Operation: File Status - Threads: 100 - Files: 1000000 pts/hadoop-1.0.x - Operation: File Status - Threads: 20 - Files: 1000000 pts/hadoop-1.0.x - Operation: Delete - Threads: 500 - Files: 100000 pts/hadoop-1.0.x - Operation: Rename - Threads: 500 - Files: 100000 pts/hadoop-1.0.x - Operation: Open - Threads: 500 - Files: 100000 pts/hadoop-1.0.x - Operation: Open - Threads: 100 - Files: 10000000 pts/hadoop-1.0.x - Operation: Open - Threads: 50 - Files: 10000000 Apache Hadoop 3.3.6 Operation: Create - Threads: 100 - Files: 100000 OpenBenchmarking.org metrics for this test profile configuration based on 93 public results since 7 September 2023 with the latest data as of 15 December 2023 .
Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.
Component
Percentile Rank
# Compatible Public Results
Ops per sec (Average)
OpenBenchmarking.org Distribution Of Public Results - Operation: Create - Threads: 100 - Files: 100000 93 Results Range From 4814 To 82508 Ops per sec 4814 6368 7922 9476 11030 12584 14138 15692 17246 18800 20354 21908 23462 25016 26570 28124 29678 31232 32786 34340 35894 37448 39002 40556 42110 43664 45218 46772 48326 49880 51434 52988 54542 56096 57650 59204 60758 62312 63866 65420 66974 68528 70082 71636 73190 74744 76298 77852 79406 80960 82514 4 8 12 16 20
Based on OpenBenchmarking.org data, the selected test / test configuration (Apache Hadoop 3.3.6 - Operation: Create - Threads: 100 - Files: 100000 ) has an average run-time of 7 minutes . By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.
OpenBenchmarking.org Minutes Time Required To Complete Benchmark Operation: Create - Threads: 100 - Files: 100000 Run-Time 4 8 12 16 20 Min: 1 / Avg: 7.08 / Max: 13
Based on public OpenBenchmarking.org results, the selected test / test configuration has an average standard deviation of 1.9% .
OpenBenchmarking.org Percent, Fewer Is Better Average Deviation Between Runs Operation: Create - Threads: 100 - Files: 100000 Deviation 4 8 12 16 20 Min: 0 / Avg: 1.89 / Max: 14
Tested CPU Architectures This benchmark has been successfully tested on the below mentioned architectures. The CPU architectures listed is where successful OpenBenchmarking.org result uploads occurred, namely for helping to determine if a given test is compatible with various alternative CPU architectures.
CPU Architecture
Kernel Identifier
Verified On
Intel / AMD x86 64-bit
x86_64
(Many Processors)
ARMv8 64-bit
aarch64
ARMv8 Neoverse-N1 128-Core
Recent Test Results
1 System - 5 Benchmark Results
AMD Ryzen 7 3800XT 8-Core - ASRock X370 Taichi - AMD Starship
NobaraLinux 39 - 6.7.6-201.fsync.fc39.x86_64 - GNOME Shell 45.4
1 System - 2 Benchmark Results
Intel Core i5-3470 - Positivo Informatica SA POS-PIQ77CL - Intel Xeon E3-1200 v2
Ubuntu 22.04 - 6.5.0-14-generic - GNOME Shell 42.9
1 System - 3 Benchmark Results
Intel Core i7-8665U - Dell 03CFV9 - Intel Cannon Point-LP
Fedora Linux 38 - 6.5.8-200.fc38.x86_64 - GNOME Shell 44.6
1 System - 2 Benchmark Results
Intel Core i7-8665U - Dell 03CFV9 - Intel Cannon Point-LP
Fedora Linux 38 - 6.5.8-200.fc38.x86_64 - GNOME Shell 44.6
2 Systems - 834 Benchmark Results
1 System - 5 Benchmark Results
AMD EPYC 7R13 48-Core - Supermicro H12SSL-I v1.02 - AMD Starship
EndeavourOS rolling - 6.6.1-zen1-1-zen - Xfce 4.18
1 System - 5 Benchmark Results
AMD EPYC 7R13 48-Core - Supermicro H12SSL-I v1.02 - AMD Starship
EndeavourOS rolling - 6.6.1-zen1-1-zen - Xfce 4.18
2 Systems - 358 Benchmark Results
AMD Ryzen 9 3900XT 12-Core - MSI MEG X570 GODLIKE - AMD Starship
Ubuntu 22.04 - 6.2.0-35-generic - GNOME Shell 42.2