I'm using mmap and re to search for words (1000+) in a file (9gig) without loading it into memory. I do this lots of times. In about 10 seconds I have 2.07 gigs of inactive memory (and 8.8mb free). Because the inactive memory goes up so fast I can't use purge. How can I prevent the inactive memory to get so high? Or is there a way to disable inactive memory while I'm running the program?
I use the following code:
def proteinID_to_uniprotID(protein):
conversion_file = open('idmapping.dat')
file_map = mmap.mmap(conversion_file.fileno(), 0, access=mmap.ACCESS_READ)
file_map.seek(0)
refseq_id_location = re.search(protein, file_map)
if refseq_id_location != None:
file_map.seek(refseq_id_location.start()-40)
uniprotID = re.findall('\\n.*'+protein+'.*\\b', file_map.read(55))[0].split('\t')[0].strip()
conversion_file.close()
file_map.close()
return uniprotID
else:
conversion_file.close()
file_map.close()
return 'not found'