Out of Memory error on Nested loop C# -
i looked , got have 1 remaining runtime error.
code following:
while ((line = reader.readline()) != null) { while (reader.peek() != '\r') { datalinestream.add(getword(reader)); } lucenedb.addupdateluceneindex(new mats_doc( datalinestream)); datalinestream.clear(); }
the code importing data loop not break , crash out due following
"an unhandled exception of type 'system.outofmemoryexception' occurred in mscorlib.dll"
what appropriate break conditions outer while loop ensure read entire file , break @ end. struggle since need advance next line , need skip first row in spreadsheet. appreciated.
*update*
i clear list of strings since creating doc lucene index , has 14 fields in , not want list large.
my code getword
private string getword(textreader inputdata) { string word = ""; while (inputdata.peek() >= 0) { word += (char)inputdata.read(); if ((word.contains(";"))) break; } return word; }
i'm guessing getword
isn't advancing reader - causing datalinestream
massive.
or file massive.
out of interest - why throwing away line
? - surely contains text you're trying read on inner loop?
update
there must logic hole somewhere in code, , guess it's either adding -1
or 0
chars list, or inner string in getword
(by way, using +=
on strings in way dreadful performance , memory usage). line terminators not \r
'naked' \n
.
that said, solve problems doing this:
string line = reader.readline(); string wordtemp; while(line != null) { string[] words = line.split(";".tochararray(), stringsplitoptions.removeemptyentries); foreach(var word in words) { wordtemp = word.trim(); //get rid of whitespace if(wordtemp.length != 0) datalinestream.add(wordtemp); } lucenedb.addupdateluceneindex(new mats_doc( datalinestream)); datalinestream.clear(); line = reader.readline(); }
as - don't throw away line read - use it, exploiting fact strips newlines out too. use string.split
crack out each of words, trimming them in process rid of trailing/leading whitespace might have.
Comments
Post a Comment