Bugzilla – Bug 336069
Memory leak when allocating many small strings/arrays.
Last modified: 2007-10-25 21:43:19 UTC
Description of Problem: Memory leak when allocating many small strings/arrays. Steps to reproduce the problem: 1. Compile and run the attached program 2. Wait until the program crashes. Actual Results: The program crashes and its memory consumption constantly climbs until the crash. Expected Results: The program should not crash and the GC should gather the unused objects. How often does this happen? Always Additional Information: The program doesn't crash under .NET CLR that comes with .NET framework 3. The memory consumption is constantly reduced by the GC and it doesn't climb up consistently like with mono. Additionally, you can see that if you replace all CreateRandomString() with CreateField() (which doesn't actually create a new string object) the program runs fine and doesn't crash on Mono. Here is the program: using System; using System.Collections.Generic; using System.Text; using System.IO; using System.Runtime.Serialization.Formatters.Binary; using System.Threading; using System.Collections; namespace LargeArrays { class Program { static void Main(string[] args) { BinaryFormatter formatter = new BinaryFormatter(); CreateRandomStrings(); while (true) { if (Console.KeyAvailable) { ConsoleKeyInfo keyInfo = Console.ReadKey(true); if (keyInfo.Key == ConsoleKey.Enter) break; } List<Hashtable> surrogates = new List<Hashtable>(); for (int i = 0; i < 200; ++i) { Hashtable surrogate = new Hashtable(); CreateLargeSurrogate(surrogate); surrogates.Add(surrogate); } byte[] bytes; using (MemoryStream memStream = new MemoryStream()) { byte[] innerBytes = null; using (MemoryStream innerStream = new MemoryStream()) { formatter.Serialize(innerStream, surrogates); innerBytes = innerStream.ToArray(); } memStream.Write(innerBytes, 0, innerBytes.Length); bytes = memStream.ToArray(); } System.Console.WriteLine(bytes.Length); Thread.Sleep(300); } } private static void CreateLargeSurrogate(Hashtable hashTable) { for (int i = 0; i < NumOfFields; ++i) { if (random.NextDouble() > 0.5) hashTable[CreateField()] = random.NextDouble(); else hashTable[CreateField()] = CreateRandomString(); } for (int i = 0; i < NumOfIntLists; ++i) { List<int> intList = new List<int>(); for (int j = 0; j < 10; ++j) intList.Add(random.Next()); hashTable[CreateField()] = intList; } for (int i = 0; i < NumOfArrayLists; ++i) { List<string> strList = new List<string>(); for (int j = 0; j < 10; ++j) strList.Add(CreateRandomString()); hashTable[CreateField()] = strList; } for (int i = 0; i < NumOfSurrogateLists; ++i) { List<Hashtable> subSurList = new List<Hashtable>(); for (int j = 0; j < 10; ++j) subSurList.Add(CreateSubSurrogate()); hashTable[CreateField()] = subSurList; } } private static Hashtable CreateSubSurrogate() { Hashtable result = new Hashtable(); for (int i = 0; i < NumOfSubSurrogateFields; ++i) { if (random.NextDouble() > 0.5) result[CreateField()] = random.NextDouble(); else result[CreateField()] = CreateRandomString(); } return result; } private static string CreateField() { int index = random.Next(0, randomStrings.Length); return randomStrings[index]; } private static void CreateRandomStrings() { for (int j = 0; j < randomStrings.Length; ++j) { string randomString = CreateRandomString(); randomStrings[j] = randomString; } } private static string CreateRandomString() { StringBuilder sb = new StringBuilder(); int len = random.Next(FieldNameMinLength, FieldNameMaxLength); char[] chars = new char[len]; for (int i = 0; i < chars.Length; ++i) { int minChar = Convert.ToInt32('A'); int maxChar = Convert.ToInt32('z'); chars[i] = Convert.ToChar(random.Next(minChar, maxChar)); } sb.Append(chars); string randomString = sb.ToString(); return randomString; } private static Random random = new Random(); private static string[] randomStrings = new string[256 * 1024]; private const int NumOfFields = 200; private const int NumOfIntLists = 5; private const int NumOfArrayLists = 5; private const int NumOfSurrogateLists = 5; private const int NumOfSubSurrogateFields = 10; private const int FieldNameMinLength = 10; private const int FieldNameMaxLength = 30; } }
It looks like a leak in the serialization infrastructure. By commenting out the serialization code the test is running "forever" on my system (SUSE 10.2, x86, Mono 1.2.5.1) consuming about 59M all the time.
But how do you explain the fact that when you use CreateField() instead of CreateRandomString() everywhere except in CreateRandomStrings() (that is, you con't generate any new strings during the program's execution) it doesn't leak and crash? Maybe the combination of both large objects (the byte arrays from serialization) and small strings causes the problem?
It's the high count of objects that seems to trigger the leak in the serailization code. Maybe it employs some static caches that linearly grow with the amount of objects being serialized so far.
This might be the cause. Also, please look here: http://www.mail-archive.com/mono-list@lists.ximian.com/msg22436.html Somebody already encountered a similar problem. Does anybody know when will it be fixed or if it has already been fixed in the latest release? (we use Mono 1.2.3.1 for our application).
Should be fixed in SVN.