Post 3: Backend Design

For the Backend phase of the project, we’ll be changing the nSIMemoryReporterManager interface and its implementation for the Linux Operating System, so that memory reports for Content Processes can also be fetched.

I shall be outlining the steps that I’ll be using for the backend in this post. Before that, a bit about Memory Reporting and Memory Reporter Managers:

We intend to periodically report the Resident Set Size (RSS), which is the portion of memory occupied by a process that resides in the RAM.  It is the number of pages the process has in real memory.  This is just the pages which count toward text, data, or stack space.  This does not include pages which have not been demand-loaded in, or which are swapped out. The other important metric that we’d measure is the Unique Set Size (USS), which is the unshared memory of the process,  that is guaranteed to be held private to the process. These measurements for RSS and USS need to be made for all content processes through a ‘Callback reporting’ function.

The ‘Memory Reporters’ do the job of measuring  the ‘amount’ of memory used by the processes. It is possible to either have a ‘Traversal- based’ reporter that traverses one or more data structures and measures the size of all the allocated blocks in the data structure or a ‘Counter-based’ reporter that maintains a counter that is incremented on each relevant allocation and decremented on each relevant de-allocation. The ‘tree-traversal’ based approach is used here.

Each reporter reports something specific – for example, how much memory is being consumed by graphics things, versus WebGL things versus JavaScript things, etc. These leaves are then summed up to report on the total memory usage.  We need to modify nsIMemoryReporterManager, and ensure that the process of getting the memory statistics is light-weight enough that we can do it every few seconds without incurring much overhead. We need to make sure that we aren’t measuring anything twice, lest we may land up into weird results. The chances of double-measuring any parameter are extremely less when using a tree data structure and a traversal-based approach. We can ‘register’ a MemoryReporter object around the piece of code whose memory we wish to measure or monitor and ‘unregister’ it (this is extremely important in order to avoid memory leaks) when we are done with the measurements. For iterating through the Content Parent, we use ‘weak pointers’ as we pass a raw reference to nsMemoryReporterManager, which
means that other pointers to that instance might go out of scope and then the nsIMemoryReporter will be deallocated, despite nsMemoryReporterManager still having a pointer to it. This not so smart ‘weak’ pointer does not keep the nsIMemoryReporter’s alive, hence we need to ‘unregister’ on completion.

In our case, we have a pretty simple hierarchy: Just the parent and the content processes. The children report back to the parent processes using the ‘callback’ method.

Let’s take a look at the callback function, which takes  in several values, including:

  • a path (which identifies the report);
  • an amount
  • a unit (most commonly bytes, but sometimes a unitless count or percentage);
  • a description of what is measured.

void callback(
in ACString process,
in AUTF8String path,
in PRInt32 kind,
in PRInt32 units,
in PRInt64 amount,
in AUTF8String description,
in nsISupports closure


We’ll be adding an extra ‘getReportsForSubprocesses’ method, which should take a
nsIMemoryReporterCallback function and would iterate through all content processes, gathering the USS and RSS metrics for each one by querying the OS, and  then calling the nsIMemoryReporterCallback with the values it has gathered for RSS and USS. The pid() accessors help us  iterate through the Content Parent.

We shall be using ‘GetResidentUnique’ (USS) and ‘GetResident’ (RSS) system calls on each of the content processes, which would then trigger ‘ResidentUniqueDistinguishedAmount’ and ‘ResidentDistinguishedAmount’ OS calls for calculating USS and RSS respectively.

At the heart of these calculations, lie the /proc/[pid]/smaps and /proc/[pid]/statm kernel specific files that are used by the GetProcSelfSmapsPrivate and GetProcSelfStatmField methods for calculating RSS and USS. These are the last and the most important functions that are called in stack by the above mentioned OS calls. The ‘amount’ returned is in bytes, which is calculated after calculating the number of pages and the page size.

Here’s a brief peek into what and how the OS does to get these metrics, something I learnt from the Linux Man Pages on kernel specific calls:                                                                                               The OS first inspects /proc/[pid]/smaps and /proc/[pid]/statm  to get an idea of the memory footprint of the process.  After clearing the ‘Referenced’ and ‘ACCESSED/YOUNG’ bits and, after waiting for some measured time interval, it once again inspects the values to get an idea of the change in the memory footprint of the process during the measured interval.  


This week is very exciting and challenging as I’m getting to work on the tricky part of the implementation and getting to learn a lot as the backend unfolds. Feel free to comment or question!





MozLondon 2016! :)


This June, I had the incredible privilege of attending Mozilla’s bi-annual (work week) meet-up, which was held at London. With more than 1350 Mozillians from all over the world, including over 150 volunteers and 64 interns, London All Hands 2016 was the perfect prolific week for all  projects across Mozilla, where  teams could meet in person and work on their future design. It began with the plenary session, opened by none other than Doctor Who followed by outstanding talks by Chris Beard and Mitchell Baker. The week followed with team-specific meet-ups and brainstorming of ideas, where every attendee could delve in and have their say. I had the pleasure of attending some amazing talks on Rust, Gecko and Servo, Google Apps Training, Electrolysis, Context Graph Data and Fireside chats with Chris Beard. With such a lot of interesting and overwhelming talks by the best speakers going on across the three splendid London hotels as venues, meeting a lot of contributors and getting to know about their work was simply inspirational, and this being my first ever conference, my experience was absolutely terrific!


Photos of the entire work week are now available at: https://www.flickr.com/photos/mozillaallhands/sets/72157669425118160/