1. 首頁
  2. 問答
  3. 錯誤貝葉斯上象夫

錯誤貝葉斯上象夫

2012-04-09 62 views
0

我跑與7演出維基備份亨利馬烏維基百科例子例子..，但測試時的分類，我收到了一個內存溢出錯誤錯誤貝葉斯上象夫

我已經粘貼下面的輸出我設定的是象夫堆大小和Java堆大小2500M

$MAHOUT_HOME/bin/mahout testclassifier -m wikipediamodel -d wikipediainput 

run with heapsize 2500 
-Xmx2500m 
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath. 



Running on hadoop, using HADOOP_HOME=/home/hduser/hadoop/hadoop 
No HADOOP_CONF_DIR set, using /home/hduser/hadoop/hadoop/conf 
MAHOUT-JOB: /home/nauman/mahout/examples/target/mahout-examples-0.7-SNAPSHOT-job.jar 
12/04/10 00:06:18 INFO common.HadoopUtil: Deleting wikipediainput-output 
12/04/10 00:06:18 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= 
12/04/10 00:06:18 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. 

Applications should implement Tool for the same. 

12/04/10 00:06:18 INFO mapred.FileInputFormat: Total input paths to process : 1 
12/04/10 00:06:18 INFO mapred.JobClient: Running job: job_local_0001 
12/04/10 00:06:18 INFO mapred.FileInputFormat: Total input paths to process : 1 
12/04/10 00:06:18 INFO mapred.MapTask: numReduceTasks: 1 
12/04/10 00:06:18 INFO mapred.MapTask: io.sort.mb = 100 
12/04/10 00:06:19 INFO mapred.MapTask: data buffer = 79691776/99614720 
12/04/10 00:06:19 INFO mapred.MapTask: record buffer = 262144/327680 
12/04/10 00:06:19 INFO bayes.BayesClassifierMapper: Bayes Parameter {basePath=wikipediamodel, classifierType=bayes, dataSource=hdfs, alpha_i=1.0, gramSize=1, verbose=false, encoding=UTF-8, confusionMatrix=null, defaultCat=unknown, testDirPath=wikipediainput} 
12/04/10 00:06:19 INFO bayes.BayesClassifierMapper: {basePath=wikipediamodel, classifierType=bayes, dataSource=hdfs, alpha_i=1.0, gramSize=1, verbose=false, encoding=UTF-8, confusionMatrix=null, defaultCat=unknown, testDirPath=wikipediainput} 
12/04/10 00:06:19 INFO bayes.BayesClassifierMapper: Testing Bayes Classifier 
12/04/10 00:06:19 INFO mapred.JobClient: map 0% reduce 0% 
12/04/10 00:06:20 INFO bayes.SequenceFileModelReader: Read 50000 feature weights 
12/04/10 00:06:20 INFO bayes.SequenceFileModelReader: Read 100000 feature weights 
12/04/10 00:06:21 INFO bayes.SequenceFileModelReader: Read 150000 feature weights 
12/04/10 00:06:21 INFO bayes.SequenceFileModelReader: Read 200000 feature weights 
12/04/10 00:06:21 INFO bayes.SequenceFileModelReader: Read 250000 feature weights 
12/04/10 00:06:21 INFO mapred.LocalJobRunner: file:/home/nauman/wikipediainput/part-r-00000:0+33554432 
12/04/10 00:06:22 INFO bayes.SequenceFileModelReader: Read 300000 feature weights 
12/04/10 00:06:22 INFO bayes.SequenceFileModelReader: Read 350000 feature weights 
12/04/10 00:06:22 INFO bayes.SequenceFileModelReader: Read 400000 feature weights 
12/04/10 00:06:22 INFO bayes.SequenceFileModelReader: Read 450000 feature weights 
12/04/10 00:06:23 INFO bayes.SequenceFileModelReader: Read 500000 feature weights 
12/04/10 00:06:23 INFO bayes.SequenceFileModelReader: Read 550000 feature weights 
12/04/10 00:06:23 INFO bayes.SequenceFileModelReader: Read 600000 feature weights 
12/04/10 00:06:23 INFO bayes.SequenceFileModelReader: Read 650000 feature weights 
12/04/10 00:06:23 INFO bayes.SequenceFileModelReader: Read 700000 feature weights 
12/04/10 00:06:23 INFO bayes.SequenceFileModelReader: Read 750000 feature weights 
12/04/10 00:06:23 INFO bayes.SequenceFileModelReader: Read 800000 feature weights 
12/04/10 00:06:23 INFO bayes.SequenceFileModelReader: Read 850000 feature weights 
12/04/10 00:06:24 INFO bayes.SequenceFileModelReader: Read 900000 feature weights 
12/04/10 00:06:25 INFO bayes.SequenceFileModelReader: Read 950000 feature weights 
12/04/10 00:06:25 INFO bayes.SequenceFileModelReader: Read 1000000 feature weights 
12/04/10 00:06:25 INFO bayes.SequenceFileModelReader: Read 1050000 feature weights 
12/04/10 00:06:26 INFO bayes.SequenceFileModelReader: Read 1100000 feature weights 
12/04/10 00:06:26 INFO bayes.SequenceFileModelReader: Read 1150000 feature weights 
12/04/10 00:06:26 INFO bayes.SequenceFileModelReader: Read 1200000 feature weights 
12/04/10 00:06:26 INFO bayes.SequenceFileModelReader: Read 1250000 feature weights 
12/04/10 00:06:26 INFO bayes.SequenceFileModelReader: Read 1300000 feature weights 
12/04/10 00:06:26 INFO bayes.SequenceFileModelReader: Read 1350000 feature weights 
12/04/10 00:06:26 INFO bayes.SequenceFileModelReader: Read 1400000 feature weights 
12/04/10 00:06:26 INFO bayes.SequenceFileModelReader: Read 1450000 feature weights 
12/04/10 00:06:26 INFO bayes.SequenceFileModelReader: Read 1500000 feature weights 
12/04/10 00:06:27 INFO bayes.SequenceFileModelReader: Read 1550000 feature weights 
12/04/10 00:06:27 INFO bayes.SequenceFileModelReader: Read 1600000 feature weights 
12/04/10 00:06:27 INFO bayes.SequenceFileModelReader: Read 1650000 feature weights 
12/04/10 00:06:27 INFO bayes.SequenceFileModelReader: Read 1700000 feature weights 
12/04/10 00:06:27 INFO bayes.SequenceFileModelReader: Read 1750000 feature weights 
12/04/10 00:06:27 INFO bayes.SequenceFileModelReader: Read 1800000 feature weights 
12/04/10 00:06:28 INFO bayes.SequenceFileModelReader: Read 1850000 feature weights 
12/04/10 00:06:28 INFO bayes.SequenceFileModelReader: Read 1900000 feature weights 
12/04/10 00:06:30 INFO bayes.SequenceFileModelReader: Read 1950000 feature weights 
12/04/10 00:06:30 INFO bayes.SequenceFileModelReader: Read 2000000 feature weights 
12/04/10 00:06:30 INFO bayes.SequenceFileModelReader: Read 2050000 feature weights 
12/04/10 00:06:30 INFO bayes.SequenceFileModelReader: Read 2100000 feature weights 
12/04/10 00:06:30 INFO bayes.SequenceFileModelReader: Read 2150000 feature weights 
12/04/10 00:06:30 INFO bayes.SequenceFileModelReader: Read 2200000 feature weights 
12/04/10 00:06:30 INFO bayes.SequenceFileModelReader: Read 2250000 feature weights 
12/04/10 00:06:30 INFO bayes.SequenceFileModelReader: Read 2300000 feature weights 
12/04/10 00:06:30 INFO bayes.SequenceFileModelReader: Read 2350000 feature weights 
12/04/10 00:06:31 INFO bayes.SequenceFileModelReader: Read 2400000 feature weights 
12/04/10 00:06:31 INFO bayes.SequenceFileModelReader: Read 2450000 feature weights 
12/04/10 00:06:31 INFO bayes.SequenceFileModelReader: Read 2500000 feature weights 
12/04/10 00:06:31 INFO bayes.SequenceFileModelReader: Read 2550000 feature weights 
12/04/10 00:06:31 INFO bayes.SequenceFileModelReader: Read 2600000 feature weights 
12/04/10 00:06:31 INFO bayes.SequenceFileModelReader: Read 2650000 feature weights 
12/04/10 00:06:31 INFO bayes.SequenceFileModelReader: Read 2700000 feature weights 
12/04/10 00:06:32 INFO bayes.SequenceFileModelReader: Read 2750000 feature weights 
12/04/10 00:06:32 INFO bayes.SequenceFileModelReader: Read 2800000 feature weights 
12/04/10 00:06:32 INFO bayes.SequenceFileModelReader: Read 2850000 feature weights 
12/04/10 00:06:32 INFO bayes.SequenceFileModelReader: Read 2900000 feature weights 
12/04/10 00:06:32 INFO bayes.SequenceFileModelReader: Read 2950000 feature weights 
12/04/10 00:06:32 INFO bayes.SequenceFileModelReader: Read 3000000 feature weights 
12/04/10 00:06:32 INFO bayes.SequenceFileModelReader: Read 3050000 feature weights 
12/04/10 00:06:33 INFO bayes.SequenceFileModelReader: Read 3100000 feature weights 
12/04/10 00:06:33 INFO bayes.SequenceFileModelReader: Read 3150000 feature weights 
12/04/10 00:06:33 INFO bayes.SequenceFileModelReader: Read 3200000 feature weights 
12/04/10 00:06:33 INFO bayes.SequenceFileModelReader: Read 3250000 feature weights 
12/04/10 00:06:33 INFO bayes.SequenceFileModelReader: Read 3300000 feature weights 
12/04/10 00:06:33 INFO bayes.SequenceFileModelReader: Read 3350000 feature weights 
12/04/10 00:06:33 INFO bayes.SequenceFileModelReader: Read 3400000 feature weights 
12/04/10 00:06:34 INFO bayes.SequenceFileModelReader: Read 3450000 feature weights 
12/04/10 00:06:34 INFO bayes.SequenceFileModelReader: Read 3500000 feature weights 
12/04/10 00:06:34 INFO bayes.SequenceFileModelReader: Read 3550000 feature weights 
12/04/10 00:06:34 INFO bayes.SequenceFileModelReader: Read 3600000 feature weights 
12/04/10 00:06:39 INFO bayes.SequenceFileModelReader: Read 3650000 feature weights 
12/04/10 00:06:39 INFO bayes.SequenceFileModelReader: Read 3700000 feature weights 
12/04/10 00:06:40 INFO bayes.SequenceFileModelReader: Read 3750000 feature weights 
12/04/10 00:06:40 INFO bayes.SequenceFileModelReader: Read 3800000 feature weights 
12/04/10 00:06:42 INFO bayes.SequenceFileModelReader: Read 3850000 feature weights 
12/04/10 00:06:42 INFO bayes.SequenceFileModelReader: Read 3900000 feature weights 
12/04/10 00:06:42 INFO bayes.SequenceFileModelReader: Read 3950000 feature weights 
12/04/10 00:06:42 INFO bayes.SequenceFileModelReader: Read 4000000 feature weights 
12/04/10 00:06:42 INFO bayes.SequenceFileModelReader: Read 4050000 feature weights 
12/04/10 00:06:44 INFO bayes.SequenceFileModelReader: Read 4100000 feature weights 
12/04/10 00:06:44 INFO bayes.SequenceFileModelReader: Read 4150000 feature weights 
12/04/10 00:06:45 INFO bayes.SequenceFileModelReader: Read 4200000 feature weights 
12/04/10 00:06:45 INFO bayes.SequenceFileModelReader: Read 4250000 feature weights 
12/04/10 00:06:45 INFO bayes.SequenceFileModelReader: Read 4300000 feature weights 
12/04/10 00:06:47 INFO bayes.SequenceFileModelReader: Read 4350000 feature weights 
12/04/10 00:06:47 INFO bayes.SequenceFileModelReader: Read 4400000 feature weights 
12/04/10 00:06:47 INFO bayes.SequenceFileModelReader: Read 4450000 feature weights 
12/04/10 00:06:47 INFO bayes.SequenceFileModelReader: Read 4500000 feature weights 
12/04/10 00:06:50 INFO bayes.SequenceFileModelReader: Read 4550000 feature weights 
12/04/10 00:06:50 INFO bayes.SequenceFileModelReader: Read 4600000 feature weights 
12/04/10 00:06:51 INFO bayes.SequenceFileModelReader: Read 4650000 feature weights 
12/04/10 00:06:51 INFO bayes.SequenceFileModelReader: Read 4700000 feature weights 
12/04/10 00:06:53 INFO bayes.SequenceFileModelReader: Read 4750000 feature weights 
12/04/10 00:06:53 INFO bayes.SequenceFileModelReader: Read 4800000 feature weights 
12/04/10 00:06:53 INFO bayes.SequenceFileModelReader: Read 4850000 feature weights 
12/04/10 00:06:56 INFO bayes.SequenceFileModelReader: Read 4900000 feature weights 
12/04/10 00:06:56 INFO bayes.SequenceFileModelReader: Read 4950000 feature weights 
12/04/10 00:06:56 INFO bayes.SequenceFileModelReader: Read 5000000 feature weights 
12/04/10 00:06:59 INFO bayes.SequenceFileModelReader: Read 5050000 feature weights 
12/04/10 00:06:59 INFO bayes.SequenceFileModelReader: Read 5100000 feature weights 
12/04/10 00:06:59 INFO bayes.SequenceFileModelReader: Read 5150000 feature weights 
12/04/10 00:07:01 INFO bayes.SequenceFileModelReader: Read 5200000 feature weights 
12/04/10 00:07:02 INFO bayes.SequenceFileModelReader: Read 5250000 feature weights 
12/04/10 00:07:02 INFO bayes.SequenceFileModelReader: Read 5300000 feature weights 
12/04/10 00:07:04 INFO bayes.SequenceFileModelReader: Read 5350000 feature weights 
12/04/10 00:07:04 INFO bayes.SequenceFileModelReader: Read 5400000 feature weights 
12/04/10 00:07:07 INFO bayes.SequenceFileModelReader: Read 5450000 feature weights 
12/04/10 00:07:07 INFO bayes.SequenceFileModelReader: Read 5500000 feature weights 
12/04/10 00:07:10 INFO bayes.SequenceFileModelReader: Read 5550000 feature weights 
12/04/10 00:07:12 INFO bayes.SequenceFileModelReader: Read 5600000 feature weights 
12/04/10 00:07:12 INFO bayes.SequenceFileModelReader: Read 5650000 feature weights 
12/04/10 00:07:15 INFO bayes.SequenceFileModelReader: Read 5700000 feature weights 
12/04/10 00:07:17 INFO bayes.SequenceFileModelReader: Read 5750000 feature weights 
12/04/10 00:07:20 INFO bayes.SequenceFileModelReader: Read 5800000 feature weights 
12/04/10 00:07:23 INFO bayes.SequenceFileModelReader: Read 5850000 feature weights 
12/04/10 00:07:25 INFO bayes.SequenceFileModelReader: Read 5900000 feature weights 
12/04/10 00:07:28 INFO bayes.SequenceFileModelReader: Read 5950000 feature weights 
12/04/10 00:07:33 INFO bayes.SequenceFileModelReader: Read 6000000 feature weights 
12/04/10 00:07:38 INFO bayes.SequenceFileModelReader: Read 6050000 feature weights 
12/04/10 00:07:46 INFO bayes.SequenceFileModelReader: Read 6100000 feature weights 
12/04/10 00:08:04 INFO bayes.SequenceFileModelReader: Read 6150000 feature weights 
12/04/10 00:08:20 INFO bayes.SequenceFileModelReader: Read 6200000 feature weights 
12/04/10 00:08:47 INFO bayes.SequenceFileModelReader: Read 6250000 feature weights 
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space 
    at sun.misc.FloatingDecimal.toJavaFormatString(FloatingDecimal.java:887) 
    at java.lang.Double.toString(Double.java:179) 
    at java.text.DigitList.set(DigitList.java:272) 
    at java.text.DecimalFormat.format(DecimalFormat.java:584) 
    at java.text.DecimalFormat.format(DecimalFormat.java:507) 
    at java.text.NumberFormat.format(NumberFormat.java:269) 
    at org.apache.hadoop.util.StringUtils.formatPercent(StringUtils.java:119) 
    at org.apache.hadoop.mapred.JobClient.monitorAndPrintJob(JobClient.java:1283) 
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1251) 
    at org.apache.mahout.classifier.bayes.mapreduce.bayes.BayesClassifierDriver.runJob(BayesClassifierDriver.java:87) 
    at org.apache.mahout.classifier.bayes.TestClassifier.classifyParallel(TestClassifier.java:288) 
    at org.apache.mahout.classifier.bayes.TestClassifier.main(TestClassifier.java:191) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
    at java.lang.reflect.Method.invoke(Method.java:597) 
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) 
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) 
    at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
    at java.lang.reflect.Method.invoke(Method.java:597) 
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156) 
12/04/10 00:17:15 WARN mapred.LocalJobRunner: job_local_0001 
java.lang.RuntimeException: Error in configuring object 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) 
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) 
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) 
Caused by: java.lang.reflect.InvocationTargetException 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
    at java.lang.reflect.Method.invoke(Method.java:597) 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) 
    ... 5 more 
Caused by: java.lang.RuntimeException: Error in configuring object 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) 
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) 
    ... 10 more 
Caused by: java.lang.reflect.InvocationTargetException 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
    at java.lang.reflect.Method.invoke(Method.java:597) 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) 
    ... 13 more 
Caused by: java.lang.OutOfMemoryError: Java heap space 
    at java.nio.HeapCharBuffer.<init>(HeapCharBuffer.java:39) 
    at java.nio.CharBuffer.allocate(CharBuffer.java:312) 
    at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:760) 
    at org.apache.hadoop.io.Text.decode(Text.java:350) 
    at org.apache.hadoop.io.Text.decode(Text.java:327) 
    at org.apache.hadoop.io.Text.toString(Text.java:254) 
    at org.apache.mahout.common.StringTuple.readFields(StringTuple.java:143) 
    at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1836) 
    at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1876) 
    at org.apache.mahout.common.iterator.sequencefile.SequenceFileIterator.computeNext(SequenceFileIterator.java:95) 
    at org.apache.mahout.common.iterator.sequencefile.SequenceFileIterator.computeNext(SequenceFileIterator.java:38) 
    at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) 
    at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) 
    at com.google.common.collect.Iterators$5.hasNext(Iterators.java:525) 
    at com.google.common.collect.ForwardingIterator.hasNext(ForwardingIterator.java:43) 
    at org.apache.mahout.classifier.bayes.SequenceFileModelReader.loadFeatureWeights(SequenceFileModelReader.java:72) 
    at org.apache.mahout.classifier.bayes.SequenceFileModelReader.loadModel(SequenceFileModelReader.java:46) 
    at org.apache.mahout.classifier.bayes.InMemoryBayesDatastore.initialize(InMemoryBayesDatastore.java:72) 
    at org.apache.mahout.classifier.bayes.ClassifierContext.initialize(ClassifierContext.java:44) 
    at org.apache.mahout.classifier.bayes.mapreduce.bayes.BayesClassifierMapper.configure(BayesClassifierMapper.java:121) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
    at java.lang.reflect.Method.invoke(Method.java:597) 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) 
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
    at java.lang.reflect.Method.invoke(Method.java:597)

來源

2012-04-09 Nauman Bashir

回答

0

我也面臨同樣的問題，有的工作了之後，我試圖通過設置亨利馬烏選擇只增加JVM的內存。試試這個：

「export MAVEN_OPTS = -Xmx1g」或者這樣會給JVM更多的內存。

嘗試併發布結果，因爲很多人都面臨着這個問題。

來源

2012-04-10 02:20:54

+0

當然，生病嘗試，讓你知道... btw維基數據約爲8 GB，即時通訊嘗試使用單個節點處理它 – 2012-04-10 09:19:30

+0

然後，我認爲減小尺寸是因爲您有單一系統的瓶頸 - 8GB文件，我覺得很難。 – 2012-04-10 12:02:25

+0

你好，我認爲這確實有所改進，但最終系統再次崩潰..我想我需要加速它在多個節點上.. – 2012-04-12 10:33:28

-1

如果你正在做它一臺機器上，減少了Hadoop的記錄大小的尺寸也可以幫助，因爲它增加的地圖任務

來源

2012-07-11 08:20:36

+1

不要這樣認爲，他的內存耗盡了映射器側載模型的內存。它不受多少整體數據進入映射器的影響。 – 2012-07-11 09:39:35

+0

如果您在單臺計算機上運行，​​請完全避免Hadoop。 Hadoop僅適用於多系統設置。 – 2012-07-18 05:40:07

1

你需要增加提供給映射器內存中的數字。將mapred.map.java.child.opts設置爲足以容納該模型的東西。

這可能是因爲你試圖加載一些不切實際的東西到內存中。

來源

2012-07-11 09:41:04

+0

我們如何確定映射器的適當內存量？ – eliasah 2014-07-04 09:29:45

相關問題

 相關問題