데이터 중심(data-driven)의 WorkFlow를 설정할 수 있는 Utillity
특징
– 경량 유틸리티 – 복잡한 설치 필요 없음
– 데이터 흐름 프로그래밍 모델 기반(DAG)
– 쉬운 학습 곡선.
– 아마존 Elastic MapReduce 지원
– MapReduce 작업뿐만 아니라 PIG Latin scripts를 실행
Hamake와 Cascading의 차이점
– Cascading는 API, hamake은 utility
Hamake는 Oozie와 Azkaban
– Oozie 및 Azkaban는 설치 및 서비스(데몬)로 실행해야 할 서버 측 시스템
HaMake 설치
1. 다운로드
|
1 |
]$ wget http://hamake.googlecode.com/files/hamake-2.0b-3.jar |
2. 설정
|
1 2 3 4 5 |
]$ vi ~/.bashrc … export HADOOP_HOME=/home/hadoop/hadoop export HADOOP_HOME_WARN_SUPPRESS=1 export PATH=$PATH:$HADOOP_HOME/bin |
export HADOOP_HOME_WARN_SUPPRESS=1 이 부분은 hadoop 명령실행시 Warning: $HADOOP_HOME is deprecated 가 발생하지 않도록 하기 위한 설정이다.
3. 실행
|
1 |
]$ hadoop jar hamake-2.0b.jar -f path_to_hamake-file |
4. Hamake Examples 실행
hamake 는 map/reduce를 hdfs에 올려놓고 실행한다.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
]$ pwd /home/hadoop/hamake/examples/bin ]$ cat start-class-size-example.sh #!/usr/bin/env bash export HAMAKE_HOME=/home/hadoop/hamake hadoop fs -rmr build hadoop fs -rmr dist hadoop fs -rmr test hadoop fs -mkdir build/lib hadoop fs -mkdir dist/examples hadoop fs -mkdir test/resources/scripts hadoop fs -put $HAMAKE_HOME/examples/*.jar dist/examples hadoop fs -put $HAMAKE_HOME/examples/data/*.jar build/lib hadoop fs -put $HAMAKE_HOME/*.jar dist hadoop fs -put $HAMAKE_HOME/examples/scripts/*.pig test/resources/scripts hadoop jar $HAMAKE_HOME/hamake-2.0b-3.jar -f file://$HAMAKE_HOME/examples/hamakefiles/class-size.xml |
HaMake 가 data-driven이라는데, 처음에는 무슨의미인지 잘이해를 못했는데,
examples을 구동해보니, 좀 감이온다.
oozie 나 azkaban의 경우는 job 간의 관계(dependencies)를 설정하는 반면,
hamake는 job간의 관계가 아닌 data 중심으로 관계를 설정한다.
예를 들면, job1의 output data의 위치를 job2의 input으로 설정해주는 방식이다.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 |
]$ cat class-size.xml <?xml version="1.0" encoding="UTF-8"?> <project name="test"> <property name="output" value="build/test"/> <property name="lib" value="build/lib"/> <property name="dist" value="dist/examples"/> <property name="scripts" value="test/resources/scripts"/> <foreach name="jar-listings"> <input> <fileset path="${lib}" mask="*.jar"/> </input> <output> <file id="jarListing" path="${output}/jar-listings/${foreach:filename}"/> </output> <mapreduce jar="${dist}/hamake-examples-${hamake:version}.jar" main="com.codeminders.hamake.examples.JarListing"> <parameter> <literal value="${foreach:path}"/> </parameter> <parameter> <reference idref="jarListing"/> </parameter> </mapreduce> </foreach> <foreach name="filter-listing"> <input> <fileset path="${output}/jar-listings"/> </input> <output> <file id="filterListing" path="${output}/jar-listings-filtered/${foreach:filename}"/> </output> <mapreduce jar="${dist}/hamake-examples-${hamake:version}.jar" main="com.codeminders.hamake.examples.JarListingFilter"> <parameter> <literal value="${foreach:path}"/> </parameter> <parameter> <reference idref="filterListing"/> </parameter> </mapreduce> </foreach> <fold name="histogram"> <input> <file id="histogramIn" path="${output}/jar-listings-filtered"/> </input> <output> <file id="histogramOut" path="${output}/class-size-histogram"/> </output> <mapreduce jar="${dist}/hamake-examples-${hamake:version}.jar" main="com.codeminders.hamake.examples.ClassSizeHistogram"> <parameter> <reference idref="histogramIn"/> </parameter> <parameter> <reference idref="histogramOut"/> </parameter> </mapreduce> </fold> <fold name="median"> <input> <file id="medianIn" path="${output}/class-size-histogram"/> </input> <output> <file id="medianOut" path="${output}/class-size-median-bin"/> </output> <pig script="${scripts}/median.pig"> <parameter name="infile"> <reference idref="medianIn"/> </parameter> <parameter name="outfile"> <reference idref="medianOut"/> </parameter> </pig> </fold> </project> |
그럼 실제로 구동을 시켜보면…
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 |
]$ ./start-class-size-example.sh Deleted hdfs://hadoop:9000/user/hadoop/build Deleted hdfs://hadoop:9000/user/hadoop/dist Deleted hdfs://hadoop:9000/user/hadoop/test 13/02/07 09:05:07 INFO hamake.Main: Using Hadoop 1.0.0 13/02/07 09:05:07 INFO hamake.Main: Working dir: hdfs://hadoop:9000/user/hadoop 13/02/07 09:05:07 INFO hamake.Main: Reading hamake-file file:///home/hadoop/hamake/examples/hamakefiles/class-size.xml 13/02/07 09:05:07 INFO syntax.SyntaxParser: Using Pig that is from classpath. 13/02/07 09:05:07 INFO hamake.TaskRunner: Starting jar-listings 13/02/07 09:05:07 INFO hamake.Utils: Downloading hdfs://hadoop:9000/user/hadoop/dist/examples/hamake-examples-2.0b-3.jar to /tmp/hamake4127449669395887859.jar 13/02/07 09:05:07 INFO hamake.Utils: Downloading hdfs://hadoop:9000/user/hadoop/dist/examples/hamake-examples-2.0b-3.jar to /tmp/hamake6347048540396342949.jar 13/02/07 09:05:07 INFO hamake.Utils: Downloading hdfs://hadoop:9000/user/hadoop/dist/examples/hamake-examples-2.0b-3.jar to /tmp/hamake6192391859646271511.jar 13/02/07 09:05:07 INFO hamake.Utils: Downloading hdfs://hadoop:9000/user/hadoop/dist/examples/hamake-examples-2.0b-3.jar to /tmp/hamake5394889691842766008.jar 13/02/07 09:05:07 INFO hamake.Utils: Downloading hdfs://hadoop:9000/user/hadoop/dist/examples/hamake-examples-2.0b-3.jar to /tmp/hamake2160185755894734071.jar 13/02/07 09:05:07 INFO hamake.Utils: Downloading hdfs://hadoop:9000/user/hadoop/dist/examples/hamake-examples-2.0b-3.jar to /tmp/hamake5877726858439379304.jar 13/02/07 09:05:07 INFO hamake.Utils: Downloading hdfs://hadoop:9000/user/hadoop/dist/examples/hamake-examples-2.0b-3.jar to /tmp/hamake5974235234289345371.jar 13/02/07 09:05:07 INFO util.NativeCodeLoader: Loaded the native-hadoop library 13/02/07 09:05:07 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:05:07 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:05:07 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:05:07 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:05:07 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:05:07 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:05:07 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:05:08 INFO mapred.FileInputFormat: Total input paths to process : 1 13/02/07 09:05:08 INFO mapred.FileInputFormat: Total input paths to process : 1 13/02/07 09:05:08 INFO mapred.FileInputFormat: Total input paths to process : 1 13/02/07 09:05:08 INFO mapred.FileInputFormat: Total input paths to process : 1 13/02/07 09:05:08 INFO mapred.FileInputFormat: Total input paths to process : 1 13/02/07 09:05:08 INFO mapred.FileInputFormat: Total input paths to process : 1 13/02/07 09:05:08 INFO mapred.FileInputFormat: Total input paths to process : 1 13/02/07 09:05:10 INFO mapred.JobClient: Running job: job_201301251753_0575 13/02/07 09:05:10 INFO mapred.JobClient: Running job: job_201301251753_0578 13/02/07 09:05:10 INFO mapred.JobClient: Running job: job_201301251753_0580 13/02/07 09:05:10 INFO mapred.JobClient: Running job: job_201301251753_0579 13/02/07 09:05:10 INFO mapred.JobClient: Running job: job_201301251753_0581 13/02/07 09:05:10 INFO mapred.JobClient: Running job: job_201301251753_0576 13/02/07 09:05:10 INFO mapred.JobClient: Running job: job_201301251753_0577 13/02/07 09:05:11 INFO mapred.JobClient: map 0% reduce 0% 13/02/07 09:05:11 INFO mapred.JobClient: map 0% reduce 0% 13/02/07 09:05:11 INFO mapred.JobClient: map 0% reduce 0% 13/02/07 09:05:11 INFO mapred.JobClient: map 0% reduce 0% 13/02/07 09:05:11 INFO mapred.JobClient: map 0% reduce 0% 13/02/07 09:05:11 INFO mapred.JobClient: map 0% reduce 0% 13/02/07 09:05:11 INFO mapred.JobClient: map 0% reduce 0% 13/02/07 09:05:25 INFO mapred.JobClient: map 100% reduce 0% 13/02/07 09:05:27 INFO mapred.JobClient: map 100% reduce 0% 13/02/07 09:05:37 INFO mapred.JobClient: map 100% reduce 100% 13/02/07 09:05:39 INFO mapred.JobClient: map 100% reduce 100% 13/02/07 09:05:42 INFO mapred.JobClient: Job complete: job_201301251753_0575 13/02/07 09:05:42 INFO mapred.JobClient: Counters: 30 13/02/07 09:05:42 INFO mapred.JobClient: Job Counters 13/02/07 09:05:42 INFO mapred.JobClient: Launched reduce tasks=1 13/02/07 09:05:42 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=15822 13/02/07 09:05:42 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/02/07 09:05:42 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/02/07 09:05:42 INFO mapred.JobClient: Launched map tasks=2 13/02/07 09:05:42 INFO mapred.JobClient: Data-local map tasks=2 13/02/07 09:05:42 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=9965 13/02/07 09:05:42 INFO mapred.JobClient: File Input Format Counters 13/02/07 09:05:42 INFO mapred.JobClient: Bytes Read=76412 13/02/07 09:05:42 INFO mapred.JobClient: File Output Format Counters 13/02/07 09:05:42 INFO mapred.JobClient: Bytes Written=3220 13/02/07 09:05:42 INFO mapred.JobClient: FileSystemCounters 13/02/07 09:05:42 INFO mapred.JobClient: FILE_BYTES_READ=3330 13/02/07 09:05:42 INFO mapred.JobClient: HDFS_BYTES_READ=77522 13/02/07 09:05:42 INFO mapred.JobClient: FILE_BYTES_WRITTEN=71219 13/02/07 09:05:42 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=3220 13/02/07 09:05:42 INFO mapred.JobClient: Map-Reduce Framework 13/02/07 09:05:42 INFO mapred.JobClient: Map output materialized bytes=3336 13/02/07 09:05:42 INFO mapred.JobClient: Map input records=66 13/02/07 09:05:42 INFO mapred.JobClient: Reduce shuffle bytes=1668 13/02/07 09:05:42 INFO mapred.JobClient: Spilled Records=104 13/02/07 09:05:42 INFO mapred.JobClient: Map output bytes=3220 13/02/07 09:05:42 INFO mapred.JobClient: Total committed heap usage (bytes)=341966848 13/02/07 09:05:42 INFO mapred.JobClient: CPU time spent (ms)=3360 13/02/07 09:05:42 INFO mapred.JobClient: Map input bytes=66 13/02/07 09:05:42 INFO mapred.JobClient: SPLIT_RAW_BYTES=228 13/02/07 09:05:42 INFO mapred.JobClient: Combine input records=0 13/02/07 09:05:42 INFO mapred.JobClient: Reduce input records=52 13/02/07 09:05:42 INFO mapred.JobClient: Reduce input groups=26 13/02/07 09:05:42 INFO mapred.JobClient: Combine output records=0 13/02/07 09:05:42 INFO mapred.JobClient: Physical memory (bytes) snapshot=465227776 13/02/07 09:05:42 INFO mapred.JobClient: Reduce output records=52 13/02/07 09:05:42 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1777811456 13/02/07 09:05:42 INFO mapred.JobClient: Map output records=52 13/02/07 09:05:44 INFO mapred.JobClient: Job complete: job_201301251753_0578 13/02/07 09:05:44 INFO mapred.JobClient: Counters: 30 13/02/07 09:05:44 INFO mapred.JobClient: Job Counters 13/02/07 09:05:44 INFO mapred.JobClient: Launched reduce tasks=1 13/02/07 09:05:44 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=16808 13/02/07 09:05:44 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/02/07 09:05:44 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/02/07 09:05:44 INFO mapred.JobClient: Launched map tasks=2 13/02/07 09:05:44 INFO mapred.JobClient: Data-local map tasks=2 13/02/07 09:05:44 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=10121 13/02/07 09:05:44 INFO mapred.JobClient: File Input Format Counters 13/02/07 09:05:44 INFO mapred.JobClient: Bytes Read=495764 13/02/07 09:05:44 INFO mapred.JobClient: File Output Format Counters 13/02/07 09:05:44 INFO mapred.JobClient: Bytes Written=18650 13/02/07 09:05:44 INFO mapred.JobClient: FileSystemCounters 13/02/07 09:05:44 INFO mapred.JobClient: FILE_BYTES_READ=19180 13/02/07 09:05:44 INFO mapred.JobClient: HDFS_BYTES_READ=496756 13/02/07 09:05:44 INFO mapred.JobClient: FILE_BYTES_WRITTEN=102925 13/02/07 09:05:44 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=18650 13/02/07 09:05:44 INFO mapred.JobClient: Map-Reduce Framework 13/02/07 09:05:44 INFO mapred.JobClient: Map output materialized bytes=19186 13/02/07 09:05:44 INFO mapred.JobClient: Map input records=292 13/02/07 09:05:44 INFO mapred.JobClient: Reduce shuffle bytes=9593 13/02/07 09:05:44 INFO mapred.JobClient: Spilled Records=524 13/02/07 09:05:44 INFO mapred.JobClient: Map output bytes=18650 13/02/07 09:05:44 INFO mapred.JobClient: Total committed heap usage (bytes)=342163456 13/02/07 09:05:44 INFO mapred.JobClient: CPU time spent (ms)=3170 13/02/07 09:05:44 INFO mapred.JobClient: Map input bytes=292 13/02/07 09:05:44 INFO mapred.JobClient: SPLIT_RAW_BYTES=230 13/02/07 09:05:44 INFO mapred.JobClient: Combine input records=0 13/02/07 09:05:44 INFO mapred.JobClient: Reduce input records=262 13/02/07 09:05:44 INFO mapred.JobClient: Reduce input groups=131 13/02/07 09:05:44 INFO mapred.JobClient: Combine output records=0 13/02/07 09:05:44 INFO mapred.JobClient: Physical memory (bytes) snapshot=459124736 13/02/07 09:05:44 INFO mapred.JobClient: Reduce output records=262 13/02/07 09:05:44 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1779679232 13/02/07 09:05:44 INFO mapred.JobClient: Map output records=262 13/02/07 09:05:58 INFO mapred.JobClient: map 100% reduce 0% 13/02/07 09:05:59 INFO mapred.JobClient: map 100% reduce 0% 13/02/07 09:06:10 INFO mapred.JobClient: map 100% reduce 100% 13/02/07 09:06:11 INFO mapred.JobClient: map 100% reduce 100% 13/02/07 09:06:15 INFO mapred.JobClient: Job complete: job_201301251753_0580 13/02/07 09:06:15 INFO mapred.JobClient: Counters: 30 13/02/07 09:06:15 INFO mapred.JobClient: Job Counters 13/02/07 09:06:15 INFO mapred.JobClient: Launched reduce tasks=1 13/02/07 09:06:15 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=15839 13/02/07 09:06:15 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/02/07 09:06:15 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/02/07 09:06:15 INFO mapred.JobClient: Launched map tasks=2 13/02/07 09:06:15 INFO mapred.JobClient: Data-local map tasks=2 13/02/07 09:06:15 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=9937 13/02/07 09:06:15 INFO mapred.JobClient: File Input Format Counters 13/02/07 09:06:15 INFO mapred.JobClient: Bytes Read=48746 13/02/07 09:06:15 INFO mapred.JobClient: File Output Format Counters 13/02/07 09:06:15 INFO mapred.JobClient: Bytes Written=1838 13/02/07 09:06:15 INFO mapred.JobClient: FileSystemCounters 13/02/07 09:06:15 INFO mapred.JobClient: FILE_BYTES_READ=1904 13/02/07 09:06:15 INFO mapred.JobClient: HDFS_BYTES_READ=49854 13/02/07 09:06:15 INFO mapred.JobClient: FILE_BYTES_WRITTEN=68427 13/02/07 09:06:15 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=1838 13/02/07 09:06:15 INFO mapred.JobClient: Map-Reduce Framework 13/02/07 09:06:15 INFO mapred.JobClient: Map output materialized bytes=1910 13/02/07 09:06:15 INFO mapred.JobClient: Map input records=40 13/02/07 09:06:15 INFO mapred.JobClient: Reduce shuffle bytes=955 13/02/07 09:06:15 INFO mapred.JobClient: Spilled Records=60 13/02/07 09:06:15 INFO mapred.JobClient: Map output bytes=1838 13/02/07 09:06:15 INFO mapred.JobClient: Total committed heap usage (bytes)=293535744 13/02/07 09:06:15 INFO mapred.JobClient: CPU time spent (ms)=3250 13/02/07 09:06:15 INFO mapred.JobClient: Map input bytes=40 13/02/07 09:06:15 INFO mapred.JobClient: SPLIT_RAW_BYTES=248 13/02/07 09:06:15 INFO mapred.JobClient: Combine input records=0 13/02/07 09:06:15 INFO mapred.JobClient: Reduce input records=30 13/02/07 09:06:15 INFO mapred.JobClient: Reduce input groups=15 13/02/07 09:06:15 INFO mapred.JobClient: Combine output records=0 13/02/07 09:06:15 INFO mapred.JobClient: Physical memory (bytes) snapshot=267399168 13/02/07 09:06:15 INFO mapred.JobClient: Reduce output records=30 13/02/07 09:06:15 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1699438592 13/02/07 09:06:15 INFO mapred.JobClient: Map output records=30 13/02/07 09:06:16 INFO mapred.JobClient: Job complete: job_201301251753_0579 13/02/07 09:06:16 INFO mapred.JobClient: Counters: 30 13/02/07 09:06:16 INFO mapred.JobClient: Job Counters 13/02/07 09:06:16 INFO mapred.JobClient: Launched reduce tasks=1 13/02/07 09:06:16 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=15774 13/02/07 09:06:16 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/02/07 09:06:16 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/02/07 09:06:16 INFO mapred.JobClient: Launched map tasks=2 13/02/07 09:06:16 INFO mapred.JobClient: Data-local map tasks=2 13/02/07 09:06:16 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=10016 13/02/07 09:06:16 INFO mapred.JobClient: File Input Format Counters 13/02/07 09:06:16 INFO mapred.JobClient: Bytes Read=268382 13/02/07 09:06:16 INFO mapred.JobClient: File Output Format Counters 13/02/07 09:06:16 INFO mapred.JobClient: Bytes Written=12384 13/02/07 09:06:16 INFO mapred.JobClient: FileSystemCounters 13/02/07 09:06:16 INFO mapred.JobClient: FILE_BYTES_READ=12686 13/02/07 09:06:16 INFO mapred.JobClient: HDFS_BYTES_READ=269312 13/02/07 09:06:16 INFO mapred.JobClient: FILE_BYTES_WRITTEN=89961 13/02/07 09:06:16 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=12384 13/02/07 09:06:16 INFO mapred.JobClient: Map-Reduce Framework 13/02/07 09:06:16 INFO mapred.JobClient: Map output materialized bytes=12692 13/02/07 09:06:16 INFO mapred.JobClient: Map input records=184 13/02/07 09:06:16 INFO mapred.JobClient: Reduce shuffle bytes=6346 13/02/07 09:06:16 INFO mapred.JobClient: Spilled Records=296 13/02/07 09:06:16 INFO mapred.JobClient: Map output bytes=12384 13/02/07 09:06:16 INFO mapred.JobClient: Total committed heap usage (bytes)=342228992 13/02/07 09:06:16 INFO mapred.JobClient: CPU time spent (ms)=2620 13/02/07 09:06:16 INFO mapred.JobClient: Map input bytes=184 13/02/07 09:06:16 INFO mapred.JobClient: SPLIT_RAW_BYTES=238 13/02/07 09:06:16 INFO mapred.JobClient: Combine input records=0 13/02/07 09:06:16 INFO mapred.JobClient: Reduce input records=148 13/02/07 09:06:16 INFO mapred.JobClient: Reduce input groups=74 13/02/07 09:06:16 INFO mapred.JobClient: Combine output records=0 13/02/07 09:06:16 INFO mapred.JobClient: Physical memory (bytes) snapshot=428703744 13/02/07 09:06:16 INFO mapred.JobClient: Reduce output records=148 13/02/07 09:06:16 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1765707776 13/02/07 09:06:16 INFO mapred.JobClient: Map output records=148 13/02/07 09:06:29 INFO mapred.JobClient: map 100% reduce 0% 13/02/07 09:06:31 INFO mapred.JobClient: map 100% reduce 0% 13/02/07 09:06:42 INFO mapred.JobClient: map 100% reduce 100% 13/02/07 09:06:44 INFO mapred.JobClient: map 100% reduce 100% 13/02/07 09:06:47 INFO mapred.JobClient: Job complete: job_201301251753_0576 13/02/07 09:06:47 INFO mapred.JobClient: Counters: 30 13/02/07 09:06:47 INFO mapred.JobClient: Job Counters 13/02/07 09:06:47 INFO mapred.JobClient: Launched reduce tasks=1 13/02/07 09:06:47 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=15727 13/02/07 09:06:47 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/02/07 09:06:47 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/02/07 09:06:47 INFO mapred.JobClient: Launched map tasks=2 13/02/07 09:06:47 INFO mapred.JobClient: Data-local map tasks=2 13/02/07 09:06:47 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=9988 13/02/07 09:06:47 INFO mapred.JobClient: File Input Format Counters 13/02/07 09:06:47 INFO mapred.JobClient: Bytes Read=287904 13/02/07 09:06:47 INFO mapred.JobClient: File Output Format Counters 13/02/07 09:06:47 INFO mapred.JobClient: Bytes Written=19482 13/02/07 09:06:47 INFO mapred.JobClient: FileSystemCounters 13/02/07 09:06:47 INFO mapred.JobClient: FILE_BYTES_READ=20128 13/02/07 09:06:47 INFO mapred.JobClient: HDFS_BYTES_READ=291924 13/02/07 09:06:47 INFO mapred.JobClient: FILE_BYTES_WRITTEN=104779 13/02/07 09:06:47 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=19482 13/02/07 09:06:47 INFO mapred.JobClient: Map-Reduce Framework 13/02/07 09:06:47 INFO mapred.JobClient: Map output materialized bytes=20134 13/02/07 09:06:47 INFO mapred.JobClient: Map input records=372 13/02/07 09:06:47 INFO mapred.JobClient: Reduce shuffle bytes=10067 13/02/07 09:06:47 INFO mapred.JobClient: Spilled Records=640 13/02/07 09:06:47 INFO mapred.JobClient: Map output bytes=19482 13/02/07 09:06:47 INFO mapred.JobClient: Total committed heap usage (bytes)=318308352 13/02/07 09:06:47 INFO mapred.JobClient: CPU time spent (ms)=3300 13/02/07 09:06:47 INFO mapred.JobClient: Map input bytes=372 13/02/07 09:06:47 INFO mapred.JobClient: SPLIT_RAW_BYTES=216 13/02/07 09:06:47 INFO mapred.JobClient: Combine input records=0 13/02/07 09:06:47 INFO mapred.JobClient: Reduce input records=320 13/02/07 09:06:47 INFO mapred.JobClient: Reduce input groups=160 13/02/07 09:06:47 INFO mapred.JobClient: Combine output records=0 13/02/07 09:06:47 INFO mapred.JobClient: Physical memory (bytes) snapshot=371122176 13/02/07 09:06:47 INFO mapred.JobClient: Reduce output records=320 13/02/07 09:06:47 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1716297728 13/02/07 09:06:47 INFO mapred.JobClient: Map output records=320 13/02/07 09:06:49 INFO mapred.JobClient: Job complete: job_201301251753_0577 13/02/07 09:06:49 INFO mapred.JobClient: Counters: 30 13/02/07 09:06:49 INFO mapred.JobClient: Job Counters 13/02/07 09:06:49 INFO mapred.JobClient: Launched reduce tasks=1 13/02/07 09:06:49 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=15807 13/02/07 09:06:49 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/02/07 09:06:49 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/02/07 09:06:49 INFO mapred.JobClient: Launched map tasks=2 13/02/07 09:06:49 INFO mapred.JobClient: Data-local map tasks=2 13/02/07 09:06:49 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=10015 13/02/07 09:06:49 INFO mapred.JobClient: File Input Format Counters 13/02/07 09:06:49 INFO mapred.JobClient: Bytes Read=200912 13/02/07 09:06:49 INFO mapred.JobClient: File Output Format Counters 13/02/07 09:06:49 INFO mapred.JobClient: Bytes Written=11316 13/02/07 09:06:49 INFO mapred.JobClient: FileSystemCounters 13/02/07 09:06:49 INFO mapred.JobClient: FILE_BYTES_READ=11642 13/02/07 09:06:49 INFO mapred.JobClient: HDFS_BYTES_READ=201826 13/02/07 09:06:49 INFO mapred.JobClient: FILE_BYTES_WRITTEN=87837 13/02/07 09:06:49 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=11316 13/02/07 09:06:49 INFO mapred.JobClient: Map-Reduce Framework 13/02/07 09:06:49 INFO mapred.JobClient: Map output materialized bytes=11648 13/02/07 09:06:49 INFO mapred.JobClient: Map input records=182 13/02/07 09:06:49 INFO mapred.JobClient: Reduce shuffle bytes=5824 13/02/07 09:06:49 INFO mapred.JobClient: Spilled Records=320 13/02/07 09:06:49 INFO mapred.JobClient: Map output bytes=11316 13/02/07 09:06:49 INFO mapred.JobClient: Total committed heap usage (bytes)=342622208 13/02/07 09:06:49 INFO mapred.JobClient: CPU time spent (ms)=2590 13/02/07 09:06:49 INFO mapred.JobClient: Map input bytes=182 13/02/07 09:06:49 INFO mapred.JobClient: SPLIT_RAW_BYTES=226 13/02/07 09:06:49 INFO mapred.JobClient: Combine input records=0 13/02/07 09:06:49 INFO mapred.JobClient: Reduce input records=160 13/02/07 09:06:49 INFO mapred.JobClient: Reduce input groups=80 13/02/07 09:06:49 INFO mapred.JobClient: Combine output records=0 13/02/07 09:06:49 INFO mapred.JobClient: Physical memory (bytes) snapshot=425394176 13/02/07 09:06:49 INFO mapred.JobClient: Reduce output records=160 13/02/07 09:06:49 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1777963008 13/02/07 09:06:49 INFO mapred.JobClient: Map output records=160 13/02/07 09:07:02 INFO mapred.JobClient: map 100% reduce 0% 13/02/07 09:07:14 INFO mapred.JobClient: map 100% reduce 100% 13/02/07 09:07:19 INFO mapred.JobClient: Job complete: job_201301251753_0581 13/02/07 09:07:19 INFO mapred.JobClient: Counters: 30 13/02/07 09:07:19 INFO mapred.JobClient: Job Counters 13/02/07 09:07:19 INFO mapred.JobClient: Launched reduce tasks=1 13/02/07 09:07:19 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=17226 13/02/07 09:07:19 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/02/07 09:07:19 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/02/07 09:07:19 INFO mapred.JobClient: Launched map tasks=2 13/02/07 09:07:19 INFO mapred.JobClient: Data-local map tasks=2 13/02/07 09:07:19 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=10031 13/02/07 09:07:19 INFO mapred.JobClient: File Input Format Counters 13/02/07 09:07:19 INFO mapred.JobClient: Bytes Read=71056 13/02/07 09:07:19 INFO mapred.JobClient: File Output Format Counters 13/02/07 09:07:19 INFO mapred.JobClient: Bytes Written=2762 13/02/07 09:07:19 INFO mapred.JobClient: FileSystemCounters 13/02/07 09:07:19 INFO mapred.JobClient: FILE_BYTES_READ=2856 13/02/07 09:07:19 INFO mapred.JobClient: HDFS_BYTES_READ=72156 13/02/07 09:07:19 INFO mapred.JobClient: FILE_BYTES_WRITTEN=70307 13/02/07 09:07:19 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=2762 13/02/07 09:07:19 INFO mapred.JobClient: Map-Reduce Framework 13/02/07 09:07:19 INFO mapred.JobClient: Map output materialized bytes=2862 13/02/07 09:07:19 INFO mapred.JobClient: Map input records=54 13/02/07 09:07:19 INFO mapred.JobClient: Reduce shuffle bytes=1431 13/02/07 09:07:19 INFO mapred.JobClient: Spilled Records=88 13/02/07 09:07:19 INFO mapred.JobClient: Map output bytes=2762 13/02/07 09:07:19 INFO mapred.JobClient: Total committed heap usage (bytes)=304152576 13/02/07 09:07:19 INFO mapred.JobClient: CPU time spent (ms)=2560 13/02/07 09:07:19 INFO mapred.JobClient: Map input bytes=54 13/02/07 09:07:19 INFO mapred.JobClient: SPLIT_RAW_BYTES=240 13/02/07 09:07:19 INFO mapred.JobClient: Combine input records=0 13/02/07 09:07:19 INFO mapred.JobClient: Reduce input records=44 13/02/07 09:07:19 INFO mapred.JobClient: Reduce input groups=22 13/02/07 09:07:19 INFO mapred.JobClient: Combine output records=0 13/02/07 09:07:19 INFO mapred.JobClient: Physical memory (bytes) snapshot=331321344 13/02/07 09:07:19 INFO mapred.JobClient: Reduce output records=44 13/02/07 09:07:19 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1694760960 13/02/07 09:07:19 INFO mapred.JobClient: Map output records=44 13/02/07 09:07:19 INFO hamake.TaskRunner: Execution of jar-listings is completed 13/02/07 09:07:19 INFO hamake.TaskRunner: Starting filter-listing 13/02/07 09:07:19 INFO hamake.Utils: Downloading hdfs://hadoop:9000/user/hadoop/dist/examples/hamake-examples-2.0b-3.jar to /tmp/hamake6034141034279588946.jar 13/02/07 09:07:19 INFO hamake.Utils: Downloading hdfs://hadoop:9000/user/hadoop/dist/examples/hamake-examples-2.0b-3.jar to /tmp/hamake7539761503636135148.jar 13/02/07 09:07:19 INFO hamake.Utils: Downloading hdfs://hadoop:9000/user/hadoop/dist/examples/hamake-examples-2.0b-3.jar to /tmp/hamake6577826317008062933.jar 13/02/07 09:07:19 INFO hamake.Utils: Downloading hdfs://hadoop:9000/user/hadoop/dist/examples/hamake-examples-2.0b-3.jar to /tmp/hamake1470897968246487913.jar 13/02/07 09:07:19 INFO hamake.Utils: Downloading hdfs://hadoop:9000/user/hadoop/dist/examples/hamake-examples-2.0b-3.jar to /tmp/hamake3694033444693921908.jar 13/02/07 09:07:19 INFO hamake.Utils: Downloading hdfs://hadoop:9000/user/hadoop/dist/examples/hamake-examples-2.0b-3.jar to /tmp/hamake2918556757311340349.jar 13/02/07 09:07:19 INFO hamake.Utils: Downloading hdfs://hadoop:9000/user/hadoop/dist/examples/hamake-examples-2.0b-3.jar to /tmp/hamake8284843469112379544.jar 13/02/07 09:07:19 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:07:19 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:07:19 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:07:19 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:07:19 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:07:19 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:07:19 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:07:20 INFO mapred.FileInputFormat: Total input paths to process : 1 13/02/07 09:07:20 INFO mapred.FileInputFormat: Total input paths to process : 1 13/02/07 09:07:20 INFO mapred.FileInputFormat: Total input paths to process : 1 13/02/07 09:07:20 INFO mapred.FileInputFormat: Total input paths to process : 1 13/02/07 09:07:20 INFO mapred.FileInputFormat: Total input paths to process : 1 13/02/07 09:07:20 INFO mapred.FileInputFormat: Total input paths to process : 1 13/02/07 09:07:20 INFO mapred.FileInputFormat: Total input paths to process : 1 13/02/07 09:07:21 INFO mapred.JobClient: Running job: job_201301251753_0582 13/02/07 09:07:22 INFO mapred.JobClient: Running job: job_201301251753_0584 13/02/07 09:07:22 INFO mapred.JobClient: Running job: job_201301251753_0585 13/02/07 09:07:22 INFO mapred.JobClient: Running job: job_201301251753_0587 13/02/07 09:07:22 INFO mapred.JobClient: Running job: job_201301251753_0588 13/02/07 09:07:22 INFO mapred.JobClient: Running job: job_201301251753_0583 13/02/07 09:07:22 INFO mapred.JobClient: Running job: job_201301251753_0586 13/02/07 09:07:22 INFO mapred.JobClient: map 0% reduce 0% 13/02/07 09:07:23 INFO mapred.JobClient: map 0% reduce 0% 13/02/07 09:07:23 INFO mapred.JobClient: map 0% reduce 0% 13/02/07 09:07:23 INFO mapred.JobClient: map 0% reduce 0% 13/02/07 09:07:23 INFO mapred.JobClient: map 0% reduce 0% 13/02/07 09:07:23 INFO mapred.JobClient: map 0% reduce 0% 13/02/07 09:07:23 INFO mapred.JobClient: map 0% reduce 0% 13/02/07 09:07:38 INFO mapred.JobClient: map 100% reduce 0% 13/02/07 09:07:38 INFO mapred.JobClient: map 100% reduce 0% 13/02/07 09:07:50 INFO mapred.JobClient: map 100% reduce 100% 13/02/07 09:07:50 INFO mapred.JobClient: map 100% reduce 100% 13/02/07 09:07:55 INFO mapred.JobClient: Job complete: job_201301251753_0582 13/02/07 09:07:55 INFO mapred.JobClient: Counters: 30 13/02/07 09:07:55 INFO mapred.JobClient: Job Counters 13/02/07 09:07:55 INFO mapred.JobClient: Launched reduce tasks=1 13/02/07 09:07:55 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=17397 13/02/07 09:07:55 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/02/07 09:07:55 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/02/07 09:07:55 INFO mapred.JobClient: Launched map tasks=2 13/02/07 09:07:55 INFO mapred.JobClient: Data-local map tasks=2 13/02/07 09:07:55 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=10026 13/02/07 09:07:55 INFO mapred.JobClient: File Input Format Counters 13/02/07 09:07:55 INFO mapred.JobClient: Bytes Read=4831 13/02/07 09:07:55 INFO mapred.JobClient: File Output Format Counters 13/02/07 09:07:55 INFO mapred.JobClient: Bytes Written=2492 13/02/07 09:07:55 INFO mapred.JobClient: FileSystemCounters 13/02/07 09:07:55 INFO mapred.JobClient: FILE_BYTES_READ=2772 13/02/07 09:07:55 INFO mapred.JobClient: HDFS_BYTES_READ=5109 13/02/07 09:07:55 INFO mapred.JobClient: FILE_BYTES_WRITTEN=70181 13/02/07 09:07:55 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=2492 13/02/07 09:07:55 INFO mapred.JobClient: Map-Reduce Framework 13/02/07 09:07:55 INFO mapred.JobClient: Map output materialized bytes=2778 13/02/07 09:07:55 INFO mapred.JobClient: Map input records=52 13/02/07 09:07:55 INFO mapred.JobClient: Reduce shuffle bytes=1398 13/02/07 09:07:55 INFO mapred.JobClient: Spilled Records=104 13/02/07 09:07:55 INFO mapred.JobClient: Map output bytes=2662 13/02/07 09:07:55 INFO mapred.JobClient: Total committed heap usage (bytes)=283115520 13/02/07 09:07:55 INFO mapred.JobClient: CPU time spent (ms)=2530 13/02/07 09:07:55 INFO mapred.JobClient: Map input bytes=3220 13/02/07 09:07:55 INFO mapred.JobClient: SPLIT_RAW_BYTES=278 13/02/07 09:07:55 INFO mapred.JobClient: Combine input records=0 13/02/07 09:07:55 INFO mapred.JobClient: Reduce input records=52 13/02/07 09:07:55 INFO mapred.JobClient: Reduce input groups=26 13/02/07 09:07:55 INFO mapred.JobClient: Combine output records=0 13/02/07 09:07:55 INFO mapred.JobClient: Physical memory (bytes) snapshot=252534784 13/02/07 09:07:55 INFO mapred.JobClient: Reduce output records=52 13/02/07 09:07:55 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1755082752 13/02/07 09:07:55 INFO mapred.JobClient: Map output records=52 13/02/07 09:07:55 INFO mapred.JobClient: Job complete: job_201301251753_0584 13/02/07 09:07:55 INFO mapred.JobClient: Counters: 30 13/02/07 09:07:55 INFO mapred.JobClient: Job Counters 13/02/07 09:07:55 INFO mapred.JobClient: Launched reduce tasks=1 13/02/07 09:07:55 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=15779 13/02/07 09:07:55 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/02/07 09:07:55 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/02/07 09:07:55 INFO mapred.JobClient: Launched map tasks=2 13/02/07 09:07:55 INFO mapred.JobClient: Data-local map tasks=2 13/02/07 09:07:55 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=10016 13/02/07 09:07:55 INFO mapred.JobClient: File Input Format Counters 13/02/07 09:07:55 INFO mapred.JobClient: Bytes Read=21614 13/02/07 09:07:55 INFO mapred.JobClient: File Output Format Counters 13/02/07 09:07:55 INFO mapred.JobClient: Bytes Written=14982 13/02/07 09:07:55 INFO mapred.JobClient: FileSystemCounters 13/02/07 09:07:55 INFO mapred.JobClient: FILE_BYTES_READ=16330 13/02/07 09:07:55 INFO mapred.JobClient: HDFS_BYTES_READ=21894 13/02/07 09:07:55 INFO mapred.JobClient: FILE_BYTES_WRITTEN=97303 13/02/07 09:07:55 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=14982 13/02/07 09:07:55 INFO mapred.JobClient: Map-Reduce Framework 13/02/07 09:07:55 INFO mapred.JobClient: Map output materialized bytes=16336 13/02/07 09:07:55 INFO mapred.JobClient: Map input records=262 13/02/07 09:07:55 INFO mapred.JobClient: Reduce shuffle bytes=8146 13/02/07 09:07:55 INFO mapred.JobClient: Spilled Records=524 13/02/07 09:07:55 INFO mapred.JobClient: Map output bytes=15800 13/02/07 09:07:55 INFO mapred.JobClient: Total committed heap usage (bytes)=343146496 13/02/07 09:07:55 INFO mapred.JobClient: CPU time spent (ms)=2580 13/02/07 09:07:55 INFO mapred.JobClient: Map input bytes=18650 13/02/07 09:07:55 INFO mapred.JobClient: SPLIT_RAW_BYTES=280 13/02/07 09:07:55 INFO mapred.JobClient: Combine input records=0 13/02/07 09:07:55 INFO mapred.JobClient: Reduce input records=262 13/02/07 09:07:55 INFO mapred.JobClient: Reduce input groups=127 13/02/07 09:07:55 INFO mapred.JobClient: Combine output records=0 13/02/07 09:07:55 INFO mapred.JobClient: Physical memory (bytes) snapshot=425299968 13/02/07 09:07:55 INFO mapred.JobClient: Reduce output records=262 13/02/07 09:07:55 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1833242624 13/02/07 09:07:55 INFO mapred.JobClient: Map output records=262 13/02/07 09:08:09 INFO mapred.JobClient: map 100% reduce 0% 13/02/07 09:08:09 INFO mapred.JobClient: map 100% reduce 0% 13/02/07 09:08:22 INFO mapred.JobClient: map 100% reduce 100% 13/02/07 09:08:22 INFO mapred.JobClient: map 100% reduce 100% 13/02/07 09:08:27 INFO mapred.JobClient: Job complete: job_201301251753_0585 13/02/07 09:08:27 INFO mapred.JobClient: Counters: 30 13/02/07 09:08:27 INFO mapred.JobClient: Job Counters 13/02/07 09:08:27 INFO mapred.JobClient: Launched reduce tasks=1 13/02/07 09:08:27 INFO mapred.JobClient: Job complete: job_201301251753_0587 13/02/07 09:08:27 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=15696 13/02/07 09:08:27 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/02/07 09:08:27 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/02/07 09:08:27 INFO mapred.JobClient: Launched map tasks=2 13/02/07 09:08:27 INFO mapred.JobClient: Data-local map tasks=2 13/02/07 09:08:27 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=10078 13/02/07 09:08:27 INFO mapred.JobClient: File Input Format Counters 13/02/07 09:08:27 INFO mapred.JobClient: Bytes Read=4144 13/02/07 09:08:27 INFO mapred.JobClient: File Output Format Counters 13/02/07 09:08:27 INFO mapred.JobClient: Bytes Written=2146 13/02/07 09:08:27 INFO mapred.JobClient: FileSystemCounters 13/02/07 09:08:27 INFO mapred.JobClient: FILE_BYTES_READ=2460 13/02/07 09:08:27 INFO mapred.JobClient: HDFS_BYTES_READ=4434 13/02/07 09:08:27 INFO mapred.JobClient: FILE_BYTES_WRITTEN=69593 13/02/07 09:08:27 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=2146 13/02/07 09:08:27 INFO mapred.JobClient: Map-Reduce Framework 13/02/07 09:08:27 INFO mapred.JobClient: Map output materialized bytes=2466 13/02/07 09:08:27 INFO mapred.JobClient: Map input records=44 13/02/07 09:08:27 INFO mapred.JobClient: Reduce shuffle bytes=1282 13/02/07 09:08:27 INFO mapred.JobClient: Spilled Records=88 13/02/07 09:08:27 INFO mapred.JobClient: Map output bytes=2366 13/02/07 09:08:27 INFO mapred.JobClient: Total committed heap usage (bytes)=342556672 13/02/07 09:08:27 INFO mapred.JobClient: CPU time spent (ms)=2970 13/02/07 09:08:27 INFO mapred.JobClient: Map input bytes=2762 13/02/07 09:08:27 INFO mapred.JobClient: SPLIT_RAW_BYTES=290 13/02/07 09:08:27 INFO mapred.JobClient: Combine input records=0 13/02/07 09:08:27 INFO mapred.JobClient: Reduce input records=44 13/02/07 09:08:27 INFO mapred.JobClient: Reduce input groups=1 13/02/07 09:08:27 INFO mapred.JobClient: Combine output records=0 13/02/07 09:08:27 INFO mapred.JobClient: Physical memory (bytes) snapshot=429936640 13/02/07 09:08:27 INFO mapred.JobClient: Reduce output records=44 13/02/07 09:08:27 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1769123840 13/02/07 09:08:27 INFO mapred.JobClient: Map output records=44 13/02/07 09:08:27 INFO mapred.JobClient: Counters: 30 13/02/07 09:08:27 INFO mapred.JobClient: Job Counters 13/02/07 09:08:27 INFO mapred.JobClient: Launched reduce tasks=1 13/02/07 09:08:27 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=15705 13/02/07 09:08:27 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/02/07 09:08:27 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/02/07 09:08:27 INFO mapred.JobClient: Launched map tasks=2 13/02/07 09:08:27 INFO mapred.JobClient: Data-local map tasks=2 13/02/07 09:08:27 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=10079 13/02/07 09:08:27 INFO mapred.JobClient: File Input Format Counters 13/02/07 09:08:27 INFO mapred.JobClient: Bytes Read=22030 13/02/07 09:08:27 INFO mapred.JobClient: File Output Format Counters 13/02/07 09:08:27 INFO mapred.JobClient: Bytes Written=15002 13/02/07 09:08:27 INFO mapred.JobClient: FileSystemCounters 13/02/07 09:08:27 INFO mapred.JobClient: FILE_BYTES_READ=16722 13/02/07 09:08:27 INFO mapred.JobClient: HDFS_BYTES_READ=22296 13/02/07 09:08:27 INFO mapred.JobClient: FILE_BYTES_WRITTEN=98045 13/02/07 09:08:27 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=15002 13/02/07 09:08:27 INFO mapred.JobClient: Map-Reduce Framework 13/02/07 09:08:27 INFO mapred.JobClient: Map output materialized bytes=16728 13/02/07 09:08:27 INFO mapred.JobClient: Map input records=320 13/02/07 09:08:27 INFO mapred.JobClient: Reduce shuffle bytes=8308 13/02/07 09:08:27 INFO mapred.JobClient: Spilled Records=640 13/02/07 09:08:27 INFO mapred.JobClient: Map output bytes=16076 13/02/07 09:08:27 INFO mapred.JobClient: Total committed heap usage (bytes)=342556672 13/02/07 09:08:27 INFO mapred.JobClient: CPU time spent (ms)=3540 13/02/07 09:08:27 INFO mapred.JobClient: Map input bytes=19482 13/02/07 09:08:27 INFO mapred.JobClient: SPLIT_RAW_BYTES=266 13/02/07 09:08:27 INFO mapred.JobClient: Combine input records=0 13/02/07 09:08:27 INFO mapred.JobClient: Reduce input records=320 13/02/07 09:08:27 INFO mapred.JobClient: Reduce input groups=158 13/02/07 09:08:27 INFO mapred.JobClient: Combine output records=0 13/02/07 09:08:27 INFO mapred.JobClient: Physical memory (bytes) snapshot=443699200 13/02/07 09:08:27 INFO mapred.JobClient: Reduce output records=320 13/02/07 09:08:27 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1777602560 13/02/07 09:08:27 INFO mapred.JobClient: Map output records=320 13/02/07 09:08:41 INFO mapred.JobClient: map 100% reduce 0% 13/02/07 09:08:42 INFO mapred.JobClient: map 100% reduce 0% 13/02/07 09:08:54 INFO mapred.JobClient: map 100% reduce 100% 13/02/07 09:08:55 INFO mapred.JobClient: map 100% reduce 100% 13/02/07 09:08:59 INFO mapred.JobClient: Job complete: job_201301251753_0588 13/02/07 09:08:59 INFO mapred.JobClient: Counters: 30 13/02/07 09:08:59 INFO mapred.JobClient: Job Counters 13/02/07 09:08:59 INFO mapred.JobClient: Launched reduce tasks=1 13/02/07 09:08:59 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=16426 13/02/07 09:08:59 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/02/07 09:08:59 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/02/07 09:08:59 INFO mapred.JobClient: Launched map tasks=2 13/02/07 09:08:59 INFO mapred.JobClient: Data-local map tasks=2 13/02/07 09:08:59 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=10012 13/02/07 09:08:59 INFO mapred.JobClient: File Input Format Counters 13/02/07 09:08:59 INFO mapred.JobClient: Bytes Read=2758 13/02/07 09:08:59 INFO mapred.JobClient: File Output Format Counters 13/02/07 09:08:59 INFO mapred.JobClient: Bytes Written=1418 13/02/07 09:08:59 INFO mapred.JobClient: FileSystemCounters 13/02/07 09:08:59 INFO mapred.JobClient: FILE_BYTES_READ=1634 13/02/07 09:08:59 INFO mapred.JobClient: HDFS_BYTES_READ=3056 13/02/07 09:08:59 INFO mapred.JobClient: FILE_BYTES_WRITTEN=67965 13/02/07 09:08:59 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=1418 13/02/07 09:08:59 INFO mapred.JobClient: Map-Reduce Framework 13/02/07 09:08:59 INFO mapred.JobClient: Map output materialized bytes=1640 13/02/07 09:08:59 INFO mapred.JobClient: Map input records=30 13/02/07 09:08:59 INFO mapred.JobClient: Reduce shuffle bytes=857 13/02/07 09:08:59 INFO mapred.JobClient: Spilled Records=60 13/02/07 09:08:59 INFO mapred.JobClient: Map output bytes=1568 13/02/07 09:08:59 INFO mapred.JobClient: Total committed heap usage (bytes)=342425600 13/02/07 09:08:59 INFO mapred.JobClient: CPU time spent (ms)=3010 13/02/07 09:08:59 INFO mapred.JobClient: Map input bytes=1838 13/02/07 09:08:59 INFO mapred.JobClient: SPLIT_RAW_BYTES=298 13/02/07 09:08:59 INFO mapred.JobClient: Combine input records=0 13/02/07 09:08:59 INFO mapred.JobClient: Reduce input records=30 13/02/07 09:08:59 INFO mapred.JobClient: Reduce input groups=1 13/02/07 09:08:59 INFO mapred.JobClient: Combine output records=0 13/02/07 09:08:59 INFO mapred.JobClient: Physical memory (bytes) snapshot=452747264 13/02/07 09:08:59 INFO mapred.JobClient: Reduce output records=30 13/02/07 09:08:59 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1834110976 13/02/07 09:08:59 INFO mapred.JobClient: Map output records=30 13/02/07 09:09:01 INFO mapred.JobClient: Job complete: job_201301251753_0583 13/02/07 09:09:01 INFO mapred.JobClient: Counters: 30 13/02/07 09:09:01 INFO mapred.JobClient: Job Counters 13/02/07 09:09:01 INFO mapred.JobClient: Launched reduce tasks=1 13/02/07 09:09:01 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=15759 13/02/07 09:09:01 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/02/07 09:09:01 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/02/07 09:09:01 INFO mapred.JobClient: Launched map tasks=2 13/02/07 09:09:01 INFO mapred.JobClient: Data-local map tasks=2 13/02/07 09:09:01 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=10688 13/02/07 09:09:01 INFO mapred.JobClient: File Input Format Counters 13/02/07 09:09:01 INFO mapred.JobClient: Bytes Read=13851 13/02/07 09:09:01 INFO mapred.JobClient: File Output Format Counters 13/02/07 09:09:01 INFO mapred.JobClient: Bytes Written=9076 13/02/07 09:09:01 INFO mapred.JobClient: FileSystemCounters 13/02/07 09:09:01 INFO mapred.JobClient: FILE_BYTES_READ=9910 13/02/07 09:09:01 INFO mapred.JobClient: HDFS_BYTES_READ=14127 13/02/07 09:09:01 INFO mapred.JobClient: FILE_BYTES_WRITTEN=84451 13/02/07 09:09:01 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=9076 13/02/07 09:09:01 INFO mapred.JobClient: Map-Reduce Framework 13/02/07 09:09:01 INFO mapred.JobClient: Map output materialized bytes=9916 13/02/07 09:09:01 INFO mapred.JobClient: Map input records=160 13/02/07 09:09:01 INFO mapred.JobClient: Reduce shuffle bytes=5002 13/02/07 09:09:01 INFO mapred.JobClient: Spilled Records=320 13/02/07 09:09:01 INFO mapred.JobClient: Map output bytes=9584 13/02/07 09:09:01 INFO mapred.JobClient: Total committed heap usage (bytes)=282263552 13/02/07 09:09:01 INFO mapred.JobClient: CPU time spent (ms)=3580 13/02/07 09:09:01 INFO mapred.JobClient: Map input bytes=11316 13/02/07 09:09:01 INFO mapred.JobClient: SPLIT_RAW_BYTES=276 13/02/07 09:09:01 INFO mapred.JobClient: Combine input records=0 13/02/07 09:09:01 INFO mapred.JobClient: Reduce input records=160 13/02/07 09:09:01 INFO mapred.JobClient: Reduce input groups=78 13/02/07 09:09:01 INFO mapred.JobClient: Combine output records=0 13/02/07 09:09:01 INFO mapred.JobClient: Physical memory (bytes) snapshot=357048320 13/02/07 09:09:01 INFO mapred.JobClient: Reduce output records=160 13/02/07 09:09:01 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1693650944 13/02/07 09:09:01 INFO mapred.JobClient: Map output records=160 13/02/07 09:09:14 INFO mapred.JobClient: map 100% reduce 0% 13/02/07 09:09:26 INFO mapred.JobClient: map 100% reduce 100% 13/02/07 09:09:31 INFO mapred.JobClient: Job complete: job_201301251753_0586 13/02/07 09:09:31 INFO mapred.JobClient: Counters: 30 13/02/07 09:09:31 INFO mapred.JobClient: Job Counters 13/02/07 09:09:31 INFO mapred.JobClient: Launched reduce tasks=1 13/02/07 09:09:31 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=15758 13/02/07 09:09:31 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/02/07 09:09:31 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/02/07 09:09:31 INFO mapred.JobClient: Launched map tasks=2 13/02/07 09:09:31 INFO mapred.JobClient: Data-local map tasks=2 13/02/07 09:09:31 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=10006 13/02/07 09:09:31 INFO mapred.JobClient: File Input Format Counters 13/02/07 09:09:31 INFO mapred.JobClient: Bytes Read=14385 13/02/07 09:09:31 INFO mapred.JobClient: File Output Format Counters 13/02/07 09:09:31 INFO mapred.JobClient: Bytes Written=10312 13/02/07 09:09:31 INFO mapred.JobClient: FileSystemCounters 13/02/07 09:09:31 INFO mapred.JobClient: FILE_BYTES_READ=11084 13/02/07 09:09:31 INFO mapred.JobClient: HDFS_BYTES_READ=14673 13/02/07 09:09:31 INFO mapred.JobClient: FILE_BYTES_WRITTEN=86835 13/02/07 09:09:31 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=10312 13/02/07 09:09:31 INFO mapred.JobClient: Map-Reduce Framework 13/02/07 09:09:31 INFO mapred.JobClient: Map output materialized bytes=11090 13/02/07 09:09:31 INFO mapred.JobClient: Map input records=148 13/02/07 09:09:31 INFO mapred.JobClient: Reduce shuffle bytes=5582 13/02/07 09:09:31 INFO mapred.JobClient: Spilled Records=296 13/02/07 09:09:31 INFO mapred.JobClient: Map output bytes=10782 13/02/07 09:09:31 INFO mapred.JobClient: Total committed heap usage (bytes)=283377664 13/02/07 09:09:31 INFO mapred.JobClient: CPU time spent (ms)=3310 13/02/07 09:09:31 INFO mapred.JobClient: Map input bytes=12384 13/02/07 09:09:31 INFO mapred.JobClient: SPLIT_RAW_BYTES=288 13/02/07 09:09:31 INFO mapred.JobClient: Combine input records=0 13/02/07 09:09:31 INFO mapred.JobClient: Reduce input records=148 13/02/07 09:09:31 INFO mapred.JobClient: Reduce input groups=74 13/02/07 09:09:31 INFO mapred.JobClient: Combine output records=0 13/02/07 09:09:31 INFO mapred.JobClient: Physical memory (bytes) snapshot=274747392 13/02/07 09:09:31 INFO mapred.JobClient: Reduce output records=148 13/02/07 09:09:31 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1754222592 13/02/07 09:09:31 INFO mapred.JobClient: Map output records=148 13/02/07 09:09:31 INFO hamake.TaskRunner: Execution of filter-listing is completed 13/02/07 09:09:31 INFO hamake.TaskRunner: Starting histogram 13/02/07 09:09:31 INFO hamake.Utils: Downloading hdfs://hadoop:9000/user/hadoop/dist/examples/hamake-examples-2.0b-3.jar to /tmp/hamake3547475163145384670.jar 13/02/07 09:09:32 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:09:32 INFO mapred.FileInputFormat: Total input paths to process : 7 13/02/07 09:09:33 INFO mapred.JobClient: Running job: job_201301251753_0589 13/02/07 09:09:34 INFO mapred.JobClient: map 0% reduce 0% 13/02/07 09:09:47 INFO mapred.JobClient: map 28% reduce 0% 13/02/07 09:09:48 INFO mapred.JobClient: map 100% reduce 0% 13/02/07 09:10:00 INFO mapred.JobClient: map 100% reduce 100% 13/02/07 09:10:05 INFO mapred.JobClient: Job complete: job_201301251753_0589 13/02/07 09:10:05 INFO mapred.JobClient: Counters: 30 13/02/07 09:10:05 INFO mapred.JobClient: Job Counters 13/02/07 09:10:05 INFO mapred.JobClient: Launched reduce tasks=1 13/02/07 09:10:05 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=35479 13/02/07 09:10:05 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/02/07 09:10:05 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/02/07 09:10:05 INFO mapred.JobClient: Launched map tasks=7 13/02/07 09:10:05 INFO mapred.JobClient: Data-local map tasks=7 13/02/07 09:10:05 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=9944 13/02/07 09:10:05 INFO mapred.JobClient: File Input Format Counters 13/02/07 09:10:05 INFO mapred.JobClient: Bytes Read=55428 13/02/07 09:10:05 INFO mapred.JobClient: File Output Format Counters 13/02/07 09:10:05 INFO mapred.JobClient: Bytes Written=180 13/02/07 09:10:05 INFO mapred.JobClient: FileSystemCounters 13/02/07 09:10:05 INFO mapred.JobClient: FILE_BYTES_READ=874 13/02/07 09:10:05 INFO mapred.JobClient: HDFS_BYTES_READ=56479 13/02/07 09:10:05 INFO mapred.JobClient: FILE_BYTES_WRITTEN=180593 13/02/07 09:10:05 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=180 13/02/07 09:10:05 INFO mapred.JobClient: Map-Reduce Framework 13/02/07 09:10:05 INFO mapred.JobClient: Map output materialized bytes=910 13/02/07 09:10:05 INFO mapred.JobClient: Map input records=1016 13/02/07 09:10:05 INFO mapred.JobClient: Reduce shuffle bytes=890 13/02/07 09:10:05 INFO mapred.JobClient: Spilled Records=124 13/02/07 09:10:05 INFO mapred.JobClient: Map output bytes=12192 13/02/07 09:10:05 INFO mapred.JobClient: Total committed heap usage (bytes)=999096320 13/02/07 09:10:05 INFO mapred.JobClient: CPU time spent (ms)=4510 13/02/07 09:10:05 INFO mapred.JobClient: Map input bytes=55428 13/02/07 09:10:05 INFO mapred.JobClient: SPLIT_RAW_BYTES=1051 13/02/07 09:10:05 INFO mapred.JobClient: Combine input records=1016 13/02/07 09:10:05 INFO mapred.JobClient: Reduce input records=62 13/02/07 09:10:05 INFO mapred.JobClient: Reduce input groups=22 13/02/07 09:10:05 INFO mapred.JobClient: Combine output records=62 13/02/07 09:10:05 INFO mapred.JobClient: Physical memory (bytes) snapshot=1033531392 13/02/07 09:10:05 INFO mapred.JobClient: Reduce output records=22 13/02/07 09:10:05 INFO mapred.JobClient: Virtual memory (bytes) snapshot=4693229568 13/02/07 09:10:05 INFO mapred.JobClient: Map output records=1016 13/02/07 09:10:05 INFO hamake.TaskRunner: Execution of histogram is completed 13/02/07 09:10:05 INFO hamake.TaskRunner: Starting median 13/02/07 09:10:05 INFO executionengine.HExecutionEngine: Connecting to hadoop file system at: hdfs://hadoop:9000 13/02/07 09:10:05 INFO executionengine.HExecutionEngine: Connecting to map-reduce job tracker at: hadoop:9001 13/02/07 09:10:05 WARN pig.PigServer: int is implicitly cast to long under LOGreaterThan Operator 13/02/07 09:10:05 WARN pig.PigServer: int is implicitly cast to long under LOMultiply Operator 13/02/07 09:10:05 WARN pig.PigServer: int is implicitly cast to long under LOGreaterThan Operator 13/02/07 09:10:05 WARN pig.PigServer: int is implicitly cast to long under LOMultiply Operator 13/02/07 09:10:05 WARN pig.PigServer: int is implicitly cast to long under LOGreaterThan Operator 13/02/07 09:10:05 WARN pig.PigServer: int is implicitly cast to long under LOMultiply Operator 13/02/07 09:10:05 WARN pig.PigServer: int is implicitly cast to long under LOGreaterThan Operator 13/02/07 09:10:05 WARN pig.PigServer: int is implicitly cast to long under LOMultiply Operator 13/02/07 09:10:07 INFO mapReduceLayer.CombinerOptimizer: Choosing to move algebraic foreach to combiner 13/02/07 09:10:07 INFO mapReduceLayer.MRCompiler$LastInputStreamingOptimizer: Rewrite: POPackage->POForEach to POJoinPackage 13/02/07 09:10:07 INFO mapReduceLayer.MultiQueryOptimizer: MR plan size before optimization: 8 13/02/07 09:10:07 INFO mapReduceLayer.MultiQueryOptimizer: Merged 0 out of total 1 splittees. 13/02/07 09:10:07 INFO mapReduceLayer.MultiQueryOptimizer: MR plan size after optimization: 8 13/02/07 09:10:07 INFO mapReduceLayer.JobControlCompiler: Setting up single store job 13/02/07 09:10:07 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:10:08 INFO mapReduceLayer.MapReduceLauncher: 0% complete 13/02/07 09:10:23 INFO mapReduceLayer.MapReduceLauncher: 6% complete 13/02/07 09:10:29 INFO mapReduceLayer.MapReduceLauncher: 12% complete 13/02/07 09:10:30 INFO mapReduceLayer.JobControlCompiler: Setting up single store job 13/02/07 09:10:30 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:10:46 INFO mapReduceLayer.MapReduceLauncher: 18% complete 13/02/07 09:10:56 INFO mapReduceLayer.MapReduceLauncher: 19% complete 13/02/07 09:11:02 INFO mapReduceLayer.MapReduceLauncher: 25% complete 13/02/07 09:11:12 INFO mapReduceLayer.JobControlCompiler: Setting up single store job 13/02/07 09:11:12 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:11:28 INFO mapReduceLayer.MapReduceLauncher: 31% complete 13/02/07 09:11:47 INFO mapReduceLayer.MapReduceLauncher: 37% complete 13/02/07 09:11:54 INFO mapReduceLayer.JobControlCompiler: Setting up single store job 13/02/07 09:11:54 INFO mapReduceLayer.JobControlCompiler: Setting identity combiner class. 13/02/07 09:11:54 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:12:11 INFO mapReduceLayer.MapReduceLauncher: 43% complete 13/02/07 09:12:23 INFO mapReduceLayer.MapReduceLauncher: 50% complete 13/02/07 09:12:32 INFO mapReduceLayer.JobControlCompiler: Setting up single store job 13/02/07 09:12:32 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:12:47 INFO mapReduceLayer.MapReduceLauncher: 56% complete 13/02/07 09:13:00 INFO mapReduceLayer.MapReduceLauncher: 62% complete 13/02/07 09:13:09 INFO mapReduceLayer.JobControlCompiler: Setting up single store job 13/02/07 09:13:09 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:13:23 INFO mapReduceLayer.MapReduceLauncher: 68% complete 13/02/07 09:13:36 INFO mapReduceLayer.MapReduceLauncher: 75% complete 13/02/07 09:13:46 INFO mapReduceLayer.JobControlCompiler: Setting up single store job 13/02/07 09:13:46 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:14:01 INFO mapReduceLayer.MapReduceLauncher: 81% complete 13/02/07 09:14:24 INFO mapReduceLayer.MapReduceLauncher: 82% complete 13/02/07 09:14:27 INFO mapReduceLayer.MapReduceLauncher: 85% complete 13/02/07 09:14:48 INFO mapReduceLayer.JobControlCompiler: Setting up single store job 13/02/07 09:14:48 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 13/02/07 09:15:28 INFO mapReduceLayer.MapReduceLauncher: 91% complete 13/02/07 09:15:34 INFO mapReduceLayer.MapReduceLauncher: 95% complete 13/02/07 09:15:56 INFO mapReduceLayer.MapReduceLauncher: 100% complete 13/02/07 09:15:56 INFO mapReduceLayer.MapReduceLauncher: Successfully stored result in: "hdfs://hadoop:9000/tmp/temp-1004992876/tmp1293898396" 13/02/07 09:15:56 INFO mapReduceLayer.MapReduceLauncher: Successfully stored result in: "hdfs://hadoop:9000/tmp/temp-1004992876/tmp1575159131" 13/02/07 09:15:56 INFO mapReduceLayer.MapReduceLauncher: Successfully stored result in: "hdfs://hadoop:9000/tmp/temp-1004992876/tmp163897091" 13/02/07 09:15:56 INFO mapReduceLayer.MapReduceLauncher: Successfully stored result in: "hdfs://hadoop:9000/tmp/temp-1004992876/tmp-1420499607" 13/02/07 09:15:56 INFO mapReduceLayer.MapReduceLauncher: Successfully stored result in: "hdfs://hadoop:9000/tmp/temp-1004992876/tmp1467066673" 13/02/07 09:15:56 INFO mapReduceLayer.MapReduceLauncher: Successfully stored result in: "hdfs://hadoop:9000/tmp/temp-1004992876/tmp1618554212" 13/02/07 09:15:56 INFO mapReduceLayer.MapReduceLauncher: Successfully stored result in: "hdfs://hadoop:9000/tmp/temp-1004992876/tmp-16848669" 13/02/07 09:15:56 INFO mapReduceLayer.MapReduceLauncher: Successfully stored result in: "hdfs://hadoop:9000/user/hadoop/build/test/class-size-median-bin" 13/02/07 09:15:56 INFO mapReduceLayer.MapReduceLauncher: Records written : 1 13/02/07 09:15:56 INFO mapReduceLayer.MapReduceLauncher: Bytes written : 0 13/02/07 09:15:56 INFO mapReduceLayer.MapReduceLauncher: Success! 13/02/07 09:15:56 INFO hamake.TaskRunner: Execution of median is completed ]$ |
