Art Of Creation - Dynamics AX Blog » System.IO

Split big text files in smaller ones in X++

Klaas Deforche — Wed, 21 Apr 2010 16:16:12 +0000

Below is a job I found useful many times, so don’t want to loose it.
Nothing special, but maybe you’ll find it useful too.

It splits an input text file into smaller files.
As there is one comment line for every line of code (almost), I’m sure you’ll figure it out :-).

static void KlForSplitFile(Args _args)
{
#File
AsciiIO inFile, outFile;
container rec;
int cnt = 0, fileNum = 0;
FileName outFileName;
FileIOPermission filePermissionOrig, outFilePermission;
Set permissionSet = new set(types::Class);

// settings begin
// input file
str _inFileName = @"C:\temp\testsplit\testsplit.csv";
// max number of lines
int _maxrec = 20000;
//settings end
;

//
// first loop to create the permission set
//

// assert permission for read
new FileIOPermission(_inFileName, #io_read).assert();

// create new asciiIo for the file we are reading in
inFile = new AsciiIo(_inFileName,#io_read);

// read file to check how many files will be created
while(inFile.read())
{
if(cnt == _maxrec || cnt == 0)
{
// 1 more file
filenum++;
// create filename
outFileName = strfmt('%1_%2.csv',
substr(_inFileName,1,strlen(_inFileName) - 4), fileNum);
// create permission
outFilePermission = new FileIOPermission(outFileName,#io_write);
// add permission to set
permissionset.add(outFilePermission);
// reset line counter
cnt = 0;
}
// 1 more line
cnt++;
}

// also add fileIn permission to set
filePermissionOrig = new FileIOPermission(_inFileName, #io_read);
permissionset.add(filePermissionOrig);
// revert permission assertion
CodeAccessPermission::revertAssert();

// assert permissions for set
CodeAccessPermission::assertMultiple(permissionset);

//
// loop file again and split
//

// reset counters
fileNum = 0;
cnt = 0;

// create new asciiIo for the file we are reading in
inFile = new AsciiIo(_inFileName,#io_read);

// read file to check how many files will be created
// read ahead
rec = inFile.read();
while(rec)
{
if(cnt == _maxrec || cnt == 0)
{
// 1 more file
filenum++;
// create filename
outFileName = strfmt('%1_%2.csv',
substr(_inFileName,1,strlen(_inFileName) - 4), fileNum);
outFile = new AsciiIO(outFileName,#io_write);
// reset line counter
cnt = 0;
}
// 1 more line
cnt++;
// write to split file
outFile.writeExp(rec);
// read next line
rec = inFile.read();
}

// revert permission assertion
CodeAccessPermission::revertAssert();

info('done');
}

Get folders and subfolders

Klaas Deforche — Fri, 08 Jan 2010 18:34:13 +0000

Someone on the Microsoft AX newsgroups asked how to get a list of directories and subdirectories given a folder. Here a job that does that:

static void KlForLoopFoldersSystemIO(Args _args)
{
int k; // counter for result loop
container dirs; // container for result
filePath path = @"C:\temp"; // input path

container getDirectories(str _dir, boolean _inclSubDirs = true)
{
container dList; // container to cast array into
int i; // counter for array loop
System.Array directories; // array for result from .NET call
#Define.Pattern("*") // input pattern: * = all
;

// assert interoppermissions for .NET interop
new InteropPermission(InteropKind::ClrInterop).assert();

// get directories using .NET interop
if(_inclSubDirs)
{
// include subdirectories
directories = System.IO.Directory::GetDirectories(_dir, #Pattern, System.IO.SearchOption::AllDirectories);
}
else
{
// only top directory
directories = System.IO.Directory::GetDirectories(_dir, #Pattern, System.IO.SearchOption::TopDirectoryOnly);
}

// loog .NET array and put the values in a container for easier use in x++
for( i=0;i<ClrInterop::getAnyTypeForObject(directories.get_Length()); i++ )
{
dList = conins(dList, conlen(dList)+1, ClrInterop::getAnyTypeForObject(directories.GetValue(i)));
}

// revert assertion to prevent multiple calls to the assert() method
CodeAccessPermission::revertAssert();

// return result container
return dList;
}
;

// get directories
dirs = getDirectories(path);

// use optional parameter to disable sub directories
// dirs = getDirectories(path, false);

// print the result
for(k=1;k<= conlen(dirs); k++)
{
// print the result to screen
info(conpeek(dirs, k));
}

// we're done
info("done");
}

Also check my post about how to loop files in a folder (method loopfilesSystemIO)

Append text from one file to an other

Klaas Deforche — Sat, 31 Oct 2009 07:28:24 +0000

This method will append all text from the original file to the destination file.
While this is a very easy task, the method shows many of the things you come across when you are working with files in AX, like:

- Using the #File macro
- Asserting FileIOPermission to be able to access files
- Asserting InteropPermission to be able to use .NET Interop
- Using a set to assert multiple permissions at once
- Using .NET Clr Interop in AX (better than winapi and winapiserver)
- Optional cleaning up after you’re done using reverAssert()

void AppendFileToFile(FileName original, FileName distination)
{
#File
FileIOPermission FileIOPermissionA = new FileIOPermission(distination, #io_append);
FileIOPermission FileIOPermissionR = new FileIOPermission(original, #io_read);
InteropPermission InteropPermission = new InteropPermission(InteropKind::ClrInterop);
Set permissionset = new set(types::Class);
;

// create permissionset
permissionset.add(FileIOPermissionA);
permissionset.add(FileIOPermissionR);
permissionset.add(InteropPermission);
// assert permissions
CodeAccessPermission::assertMultiple(permissionset);
// append text from source file to destination file
System.IO.File::AppendAllText(distination, System.IO.File::ReadAllText(original));

// limit the scope of the assert
CodeAccessPermission::revertAssert();
}

Because all permissions are taken care of, this code will run client side as well as server side.

WinAPI, RPC 1702 and FindFirstFileW

Klaas Deforche — Wed, 08 Apr 2009 21:39:50 +0000

A very common scenario when doing interfaces with other applications, is to have a directory where one application places files, and the other one picks those up and processes them.
Let’s say some application want to interface with Dynamics AX, one way, from that application to AX. That application will drop files of a specific format, say *.txt, in a directory. You’ll want to have a batch job in AX that checks that directory every 5 minutes for those files. When your batch job finds those files, it will open them and do something with that data, then move them to an other directory, so you don’t process them twice.

Now the first thing you’ll probably think of (well, I did…), is to use the class WinAPI. This class has some useful static methods for checking if folders or files exist, finding files with a filename of a certain format. Just the kind of thing we’re looking for. Your code might look like this:

static void loopfilesWinApi(Args _args)
{
str fileName;
int handle;
str format = "*.txt";
;

if(WinApi::folderExists("C:\\Temp\\"))
{
[handle,fileName] = WinApi::findFirstFile("C:\\Temp\\" + format);
while(fileName)
{
//currentfileName = fileName;
info(filename);
filename = WinApi::findNextFile(handle);
}
WinApi::closeHandle(handle);
}
}

That’s perfect. It checks a directory, in this cas C:\temp, for files with the extension txt… until you try to run this in batch. Your batch will have the status error, and there will be an entry in the event log saying:

RPC error: RPC exception 1702 occurred in session xxx

That error message is very misleading, but Florian Dittgen has a nice blog entry about this, that explains why this goes wrong: WinAPI methods a static and run on client. In batch, you can not use these client methods.
But hey! There is a class called WinAPIServer, surely this runs on server, right? Correct, but a lot of the methods available in WinAPI are missing in WinAPIServer. You can however extend the WinAPI server class by copying methods from the WinAPI class and modifying them a bit. Let’s try that.

This is what the WinAPI::FindFirstFile() method looks like:

client static container findFirstFile(str filename)
{
Binary data = new Binary(592); // size of WIN32_FIND_DATA when sizeof(TCHAR)==2
DLL _winApiDLL = new DLL(#KernelDLL);
DLLFunction _findFirstFile = new DLLFunction(_winApiDLL, 'FindFirstFileW');

_findFirstFile.returns(ExtTypes::DWord);

_findFirstFile.arg(ExtTypes::WString,ExtTypes::Pointer);

return [_findFirstFile.call(filename, data),data.wString(#offset44)];
}

As you can see, this method uses the function FindFirstFileW from the Kernel32 dll located in the folder C:\WINDOWS\system32. Let’s copy this method to WinAPIServer and modify it to look like this:

server static container findFirstFile(str filename)
{
#define.KernelDLL('KERNEL32')
#define.offset44(44)

InteropPermission interopPerm;
Binary data;
DLL _winApiDLL;
DLLFunction _findFirstFile;
;

interopPerm = new InteropPermission(InteropKind::DllInterop);
interopPerm.assert();

data = new Binary(592); // size of WIN32_FIND_DATA when sizeof(TCHAR)==2
_winApiDLL = new DLL(#KernelDLL);
_findFirstFile = new DLLFunction(_winApiDLL, 'FindFirstFileW');

_findFirstFile.returns(ExtTypes::DWord);

_findFirstFile.arg(ExtTypes::WString,ExtTypes::Pointer);

return [_findFirstFile.call(filename, data),data.wString(#offset44)];
}

Three things have changed here:

“Client” has been change to “Server” so it can run on server;
The code has been changed to use the InteropPermission class; this is required because the code runs on server;
Initialization of the variables is moved to after assertion of the InteropPermission.

When you test this in batch (running on server), this will work, and you can do this for all WinAPI methods in the examples above… until you try this on a x64 architecture. You’ll receive a message saying that an error occurred when calling the FindFirstFileW function in the Kernel32 DLL (the exact message slipped my mind).
The problem is described on the Axapta Programming Newsgroup. You can code your way around this problem, but I think the message is clear: don’t use WinAPI.

Instead, use the .NET Framework System.IO namespace (watch out for lowercase/uppercase here), in our case in specific, System.IO.File and System.IO.Directory.
I get this nice code example from Greg Pierce’s blog (modified it a bit):

static void loopfilesSystemIO(Args _args)
{
// loop all files that fit the required format
str fileName;
InteropPermission interopPerm;

System.Array files;
int i, j;
container fList;
;

interopPerm = new InteropPermission(InteropKind::ClrInterop);
interopPerm.assert();

files = System.IO.Directory::GetFiles(@"C:\temp", "*.txt");
for( i=0; i<ClrInterop::getAnyTypeForObject(files.get_Length()); i++ )
{
fList = conins(fList, conlen(fList)+1, ClrInterop::getAnyTypeForObject(files.GetValue(i)));
}

for(j=1;j<= conlen(fList); j++)
{
fileName = conpeek(fList, j);
info(fileName);
}
}

This will run on client and on server without any problems. To have clean code, you could create your own class, just like WinAPI, but using the System.IO namespace. That way, you can reuse this code and save time.

Conclusion: abandon WinAPI, and use System.IO instead.