r - Memory management / cannot allocate vector of size n Mb -

-

i running issues trying use large objects in r. example:

> memory.limit(4000) > = matrix(na, 1500000, 60) > = matrix(na, 2500000, 60) > = matrix(na, 3500000, 60) error: cannot allocate vector of size 801.1 mb > = matrix(na, 2500000, 60) error: cannot allocate vector of size 572.2 mb # can't go smaller anymore > rm(list=ls(all=true)) > = matrix(na, 3500000, 60) # works > b = matrix(na, 3500000, 60) error: cannot allocate vector of size 801.1 mb # there room

i understand related difficulty of obtaining contiguous blocks of memory (from here):

error messages beginning cannot allocate vector of size indicate failure obtain memory, either because size exceeded address-space limit process or, more likely, because system unable provide memory. note on 32-bit build there may enough free memory available, not large enough contiguous block of address space map it.

how can around this? main difficulty point in script , r can't allocate 200-300 mb object... can't pre-allocate block because need memory other processing. happens when dilligently remove unneeded objects.

edit: windows xp sp3, 4gb ram, r 2.12.0:

> sessioninfo() r version 2.12.0 (2010-10-15) platform: i386-pc-mingw32/i386 (32-bit)  locale: [1] lc_collate=english_caribbean.1252  lc_ctype=english_caribbean.1252    [3] lc_monetary=english_caribbean.1252 lc_numeric=c                       [5] lc_time=english_caribbean.1252      attached base packages: [1] stats     graphics  grdevices utils     datasets  methods   base

consider whether need data explicitly, or can matrix sparse? there support in r (see matrix package e.g.) sparse matrices.

keep other processes , objects in r minimum when need make objects of size. use gc() clear unused memory, or, better only create object need in 1 session.

if above cannot help, 64-bit machine ram can afford, , install 64-bit r.

if cannot there many online services remote computing.

if cannot memory-mapping tools package ff (or bigmemory sascha mentions) build new solution. in limited experience ff more advanced package, should read high performance computing topic on cran task views.


Comments

Post a Comment

Popular posts from this blog

c# - How to set Z index when using WPF DrawingContext? -

-
how set z-index drawing object when using drawingcontext.drawxxx() methods? the object drawn last have higher z index. can't change index of drawn objects. way draw in order. if using wpf (as placed tag), can use, example, canvas control. create shapes need like polyline obj = new polyline(); //... // ... set properties of obj and add them canvas uielementcollection: yourcanvasname.children.add(obj); //or yourcanvasname.children.insert(i, obj); first items of collection have higher z index. advantages in way: no need redraw on window changes, can anytime move objects , change order.
Read more

razor - Is this a bug in WebMatrix PageData? -

-
i think may have found bug in webmatrix's pagedata, not sure. concerns how pass data partial page calling page. in webmatrix documentation (tutorials, e.g. " 3 - creating consistent look ", , example code), pagedata recommended mechanism pass data between pages (e.g. content page layout page, or partial page). however have found not work other way, pass data partial page calling page. modifying or adding entries in pagedata in partial page, not seem calling page. cutting right down simplest possible example, in test page may have this: @{ pagedata["test"] = "initial entry"; } <p>before calling partial page, test value @pagedata["test"]</p> @renderpage("_testpartial.cshtml") <p>after returning calling page, test value @pagedata["test"]</p> and in _testpartial.cshtml page might have this: @{ pagedata["test"] = "modified entry"; } <p>in partial page...
Read more

visual c++ - Using relative values in array sorting ( asm ) -

-
i need sort through array , sort each individual row in array in ascending order. doesn't seem going (surprise!) keep getting hit 2 errors: a2101: cannot add 2 relocatable labels , a2026: constant expected here's sort, makes sense me, think i'm still trying implement high-lvl language techniqes assembly. there way around not being able use relative values? (the array 7 rows 9 columns, btw). mov cx, 7; cx = number of rows outer: ; outer loop walk through rows push cx mov cx, 9 mov row, cx ;rows middle: ; middle-loop walk through columns push cx sub cx, 1 ;cx = cx-1 mov column, cx ;columns inner: ;inner loop - compare , exchange column values cmp marray[row*9 + column], marray[row*9 + column+1] xchg marray[row*9 + column+1], marray[row*9 + column] ; compare , exchange values marray table inc column loop inner pop cx loop mid...
Read more