import - Make function and apply to read data in R? -

i have set of data (around 50000 data. , each 1 of them 1.5 mb). so, load data , process data first have used code;

data <- list() # creates list listcsv <- dir(pattern = "*.txt") # creates list of csv files in directory 

then use loop load each data;

for (k in 1:length(listcsv)){ data[[k]]<- read.csv(listcsv[k],sep = "",as.is = true, comment.char = "",    skip=37);                                                                                                                                                        my<- as.matrix(as.double(data[[k]][1:57600,2])); mesh <- array(my, dim = c(120,60,8)); ms<-1350*10^3    # a/m asd2=(mesh[70:75,24:36 ,2])/ms;     # in a/m  ort_my<- mean(asd2);  print(ort_my);  a[k]<-ort_my;  write(a,file="d:/ddd/ads.txt",sep='\t',ncolumns=1)} 

so, set program run if after 6 hours didn't finished. although have decent pc 32 gb ram , 6 core cpu.

i have searched forum , maybe fread function helpful people say. examples found far deal single file reading fread function.

can 1 suggest me solution of problem faster loop read data , process these many rows , columns?

i guessing there has way make extraction of want more efficient. think running in parallel save bunch of time. , save memory not storing each file.

library("data.table")  #create function want loop through in parallel readfiles <- function(x) {    data <- fread(x,skip=37)    <- as.matrix(data[1:57600,2,with=f]);    mesh <- array(my, dim = c(120,60,8));    ms<-1350*10^3    # a/m    asd2=(mesh[70:75,24:36 ,2])/ms;     # in a/m     ort_my<- mean(asd2);    return(ort_my) }   #r code run functions in parallel  library(“foreach”);library(“parallel”);library(“domc”) detectcores() #this tell how many cores available registerdomc(8) #register parallel backend  #can change .combine rbind list outputlist <- foreach(listcsv,.combine=rbind,.packages=c(”data.table”)) %dopar% (readfiles(x))  registerdoseq() #very important close out parallel backend.