Working with data

  • Revision slug: Mozilla/js-ctypes/Using_js-ctypes/Working_with_data
  • Revision title: Working with data
  • Revision id: 98271
  • Created:
  • Creator: Sheppy
  • Is current revision? No
  • Comment add some 64-bit integer info; 329 words added, 10 words removed

Revision Content

{{ gecko_minversion_header("2.0") }}{{ draft() }}

Creating CData objects

Data types for use with js-ctypes are represented by CType objects. These are JavaScript constructors; as such, they're callable functions that you can use to create new CData objects of that type. There are several ways you can go about creating new CData objects.

Creating uninitialized CData objects

There are three forms of the syntax for creating CData objects without immediately assigning them a value:

var myCDataObj = new type;

var myCDataObj = new type();

var myCDataObj = type();

These all do the same thing: they return a new CData object of the specified type, whose data buffer has been populated entirely with zeroes.

Note: If type.size is undefined, creating a new object this way will throw a TypeError exception.

Creating initialized CData objects

Similarly, you can initialize CData objects with specific values at the type of creation by specifying them as a parameter when calling the CType's constructor, like this:

var myCDataObj = new type(value);

var myCDataObj = type(value);

If the size of the specified type isn't undefined, the specified value is converted to the given type. If the conversion isn't possible, TypeError is thrown. The resulting data is then copied into a new CData object. If the original value is already a CData object, the original object is simply duplicated directly into the new one.

If type is an array type of unspecified length, the following steps are taken:

  • If the value is a size value, a new array of that length is created, with its cells ready to accept values of the same type as those in the specified array. This is the same as new ArrayType(type.elementType, value).
  • If the type represents a JavaScript string (that is, an array of jschar characters followed by a null terminator), a copy of that string is created and returned.
  • If the type is an array of 8-bit characters and value is a UTF-16 string, the new CData object is the result of converting the UTF-16 string to UTF-8, with a null terminator.
  • If the value is a JavaScript array object and it has a non-negative length, a new array is created and the contents of the array specified by value are converted to CData objects and copied into the new array, which is then returned.
  • If none of these conditions are met, a TypeError is thrown.

If type is ctypes.void_t, a TypeError is thrown.

Example: Creating an array

let arrayType = ctypes.ArrayType(ctypes.int32_t);
let myArray = new arrayType(5); 

At this point, myArray.length is 5; there are 5 entries in the array. myArray.constructor.size is 20; the total size of the array's data buffer is 20 bytes (5 entries, 4 bytes apiece).

Type casting

You can type cast data from one type to another by using the ctypes.cast() function:

var newObj = ctypes.cast(origObj, newType);

This will return a new object whose data block is shared with the original object, but whose type is newType. If the size of the new type is undefined or larger than the size of the original object's data block, TypeError is thrown.

This works very much like a standard C type cast or C++ reinterpret_cast.

Data and pointers

A CData object represents a C value in memory. You can always get a pointer to the C value using the CData object's address() method.

Objects can share memory

It's important to keep in mind that two (or more) CData objects can share the same memory block for their contents. This will happen, for example, when type casting. This is called aliasing. The shared memory can be whole or in part.

For example:

const Point = new ctypes.StructType("Point", [[ctypes.int32_t, 'x'], [ctypes.int32_t, 'y']]);
const Rect = new ctypes.StructType("Rect", [[Point, 'topLeft'], [Point, 'bottomRight']]);

var r = Rect();
var p = r.topLeft;
r.topLeft.x = 100; 

At this point, p is a reference to the topLeft field in the Rect named r. Setting the value of p.x will affect the value of r.topLeft.x, as expected.

Quirks in equality

Equality doesn't work the same way in JavaScript as it does in C, which means certain operations might not work the way you expect. In particular, comparing two different objects that are represented under-the-hood as JavaScript objects using the == or === operators will always return false. This affects comparisons of pointers, integers that are the same size as pointers, and 64-bit integers.

You can work around this by serializing these values using the toString() method and comparing using the resulting string.

See Determining if two pointers are equal for an example of how this works when comparing pointers.

Example: Checking the value of an integer

If, for example, you need to see if the value of an integer is 5, you can do so like this:

var t = ctypes.int32_t(5);

if (t.toString() == "ctypes.int32_t(5)") {
  // it's 5
} 

Working with strings

C functions expect strings to be arrays of characters, with the end of the string indicated by a null character. JavaScript, on the other hand, uses the String object to represent strings.

Converting C strings to JavaScript

The CData object provides the readString() method, which reads bytes from the specified string and returns a new JavaScript String object representing that string.

Note: The source C string is assumed to be UTF-8, and is assumed to be null terminated. If you need to convert a string that doesn't meet these requirements, you'll need to do it yourself.

For example:

var jsString = timeStr.readString();

Converting JavaScript strings to C

Happily, converting JavaScript strings to C formatted strings is easy; just create a character array containing the JavaScript string:

var myUTF8String = ctypes.char.array()("Original string.");

This creates a UTF-8 format null-terminated string in the character array named myUTF8String.

If you need a UTF-16 string, you can do this:

var myUTF16String = ctypes.jschar.array()("Original string.");
Note: At this time, there's no way to specify a particular encoding; you may only retrieve the string in UTF-8 or UTF-16 as shown above.

Using strings with C functions

You don't even need to convert strings when using them as input parameters to C functions. They get converted automatically for you. Just pass in the JavaScript String object.

However, when C functions return, they still return a char.ptr or jschar.ptr (that is, a pointer to an 8-bit or 16-bit array of characters). You'll have to convert those yourself, as covered above.

Working with pointers

Reading data referenced by pointer

This example creates a pointer to an integer, then looks at the pointed-to data using the PointerType object's contents property.

var i = ctypes.int32_t(9);    // Create a C integer whose value is 9

var p = i.address();          // Create a pointer to the integer variable i

if (p.contents == 9) {        // Look at the contents of the pointer
  // the value is 9
} else {
  // the value isn't 9
}

Setting data referenced by pointer

You can also use the contents property to set the value pointed to by a variable.

var i = ctypes.int32_t(9);    // Create a C integer variable whose value is 9

var p = i.address();          // Get a pointer to i
p.contents = 12;              // Change the value of i to 12

Checking to see if a pointer is null

This example demonstrates the use of the {{ manch("isNull") }} method to determine whether or not a pointer is null.

var p = someCDataObject.address();

if (p.isNull()) {
  // the pointer is null
} else {
  // the pointer isn't null
}

Determining if two pointers are equal

Due to quirks in how equality is determined in JavaScript, the best way to see if two pointers are equal is to convert them to strings, then compare the strings.

if (i.address().toString() == ctypes.int32_t.ptr(5).toString()) {
  // the integer i's address is 5
}

The example above not only compares the addresses, but also the type. So while the above comparison succeeds if the address of i is 5, it also only succeeds if i is in fact of type ctypes.int32_t.

If you don't care about type equality, and simply want to compare two addresses, you can use type casting:

if (ctypes.cast(p, ctypes.uintptr_t.value.toString() == "5") {
  // the pointer p's address is 5
}

This casts the pointer to a ctypes.uintptr_t, for which the value property returns a ctypes.UInt64. Calling toString() on that returns the pointer as a base 10 integer.

64-bit integers

While most numeric types in js-ctypes are represented by standard JavaScript Number objects, 64-bit integers cannot all be represented accurately by this type. For that reason, 64-bit and pointer-sized C values of numeric types don't get automatically converted to JavaScript numbers. Instead, they're converted to JavaScript objects that you can manipulate using the methods provided by the Int64 and UInt64 objects.

Note: These 64-bit integer types are intentionally low on features, so that when JavaScript adds a "big number" type, we can easily upgrade to support that properly.

Creating 64-bit integer objects

To create a new 64-bit number object, use the ctypes.Int64.Int64() method to create a new 64-bit signed integer, or the ctypes.UInt64.UInt64() method to create a new unsigned 64-bit integer. For example:

var num1 = ctypes.UInt64(5000);
var num2 = ctypes.Int64(-42);

If you need to create a 64-bit integer whose value can't be represented by a JavaScript Number, you have two options. You can build the new number using the high and low 32-bit numbers, or you can create it using a string representation.

Creating a 64-bit value using a string

You can simply pass a string representation of a 64-bit number into the Int64 or UInt64 constructor, like this:

var num1 = ctypes.Int64("400000000000");
var num2 = ctypes.Unt64("-0x1234567890ABCDEF");

As you can see, you can use this technique with both decimal and hexadecimal source strings.

Creating a 64-bit value using join()

The join() method offered by the Int64 and UInt64 objects provides another way to construct 64-bit integers. It accepts as its input parameters the high and low 32-bit values and returns a new 64-bit integer. For example:

var num = ctypes.UInt64.join(-0x12345678, 0x90ABCDEF);

Performing arithmetic with 64-bit values

The Int64 and UInt64 objects don't provide any methods for performing arithmetic, which means you'll have to do it yourself by pulling out the high and low 32-bit portions and doing math on them, then joining them back together if necessary to get the complete result. You'll also have to handle carry from the low to high word and back as appropriate.

more to come...

See also

Revision Source

<p>{{ gecko_minversion_header("2.0") }}{{ draft() }}</p>
<h2>Creating CData objects</h2>
<p>Data types for use with js-ctypes are represented by <a href="/en/js-ctypes/js-ctypes_reference/CType" title="en/js-ctypes/js-ctypes reference/CType"><code>CType</code></a> objects. These are JavaScript constructors; as such, they're callable functions that you can use to create new <a href="/en/js-ctypes/js-ctypes_reference/CData" title="en/js-ctypes/js-ctypes reference/CData"><code>CData</code></a> objects of that type. There are several ways you can go about creating new <code>CData</code> objects.</p>
<h3>Creating uninitialized CData objects</h3>
<p>There are three forms of the syntax for creating <code>CData</code> objects without immediately assigning them a value:</p>
<p><code>var myCDataObj = new <em>type</em>;</code></p>
<p><code>var myCDataObj = new <em>type</em>();</code></p>
<p><code>var myCDataObj = <em>type</em>();</code></p>
<p>These all do the same thing: they return a new <code>CData</code> object of the specified type, whose data buffer has been populated entirely with zeroes.</p>
<div class="note"><strong>Note:</strong> If <code><em>type</em>.size</code> is undefined, creating a new object this way will throw a <code>TypeError</code> exception.</div>
<h3>Creating initialized CData objects</h3>
<p>Similarly, you can initialize <code>CData</code> objects with specific values at the type of creation by specifying them as a parameter when calling the <a href="/en/js-ctypes/js-ctypes_reference/CType" title="en/js-ctypes/js-ctypes reference/CType"><code>CType</code></a>'s constructor, like this:</p>
<p><code>var myCDataObj = new <em>type</em>(<em>value</em>);</code></p>
<p><code>var myCDataObj = <em>type</em>(<em>value</em>);</code></p>
<p>If the size of the specified type isn't undefined, the specified value is converted to the given type. If the conversion isn't possible, <code>TypeError</code> is thrown. The resulting data is then copied into a new <code>CData</code> object. If the original value is already a <code>CData</code> object, the original object is simply duplicated directly into the new one.</p>
<p>If <em><code>type</code></em> is an array type of unspecified length, the following steps are taken:</p>
<ul> <li>If the <code><em>value</em></code> is a size value, a new array of that length is created, with its cells ready to accept values of the same type as those in the specified array. This is the same as <code>new ArrayType(<em>type</em>.elementType, <em>value</em>)</code>.</li> <li>If the <em><code>type</code></em> represents a JavaScript string (that is, an array of jschar characters followed by a null terminator), a copy of that string is created and returned.</li> <li>If the <em><code>type</code></em> is an array of 8-bit characters and <em><code>value</code></em> is a UTF-16 string, the new <code>CData</code> object is the result of converting the UTF-16 string to UTF-8, with a null terminator.</li> <li>If the <code><em>value</em></code> is a JavaScript array object and it has a non-negative length, a new array is created and the contents of the array specified by <code><em>value</em></code> are converted to <code>CData</code> objects and copied into the new array, which is then returned.</li> <li>If none of these conditions are met, a <code>TypeError</code> is thrown.</li>
</ul>
<p>If <em><code>type</code></em> is <code>ctypes.void_t</code>, a <code>TypeError</code> is thrown.</p>
<h4>Example: Creating an array</h4>
<pre class="brush: js">let arrayType = ctypes.ArrayType(ctypes.int32_t);
let myArray = new arrayType(5); 
</pre>
<p>At this point, <code>myArray.length</code> is 5; there are 5 entries in the array. <code>myArray.constructor.size</code> is 20; the total size of the array's data buffer is 20 bytes (5 entries, 4 bytes apiece).</p>
<h2>Type casting</h2>
<p>You can type cast data from one type to another by using the <code>ctypes.cast()</code> function:</p>
<p><code>var newObj = ctypes.cast(<em>origObj</em>, <em>newType</em>);</code></p>
<p>This will return a new object whose data block is shared with the original object, but whose type is <code><em>newType</em></code>. If the size of the new type is undefined or larger than the size of the original object's data block, <code>TypeError</code> is thrown.</p>
<p>This works very much like a standard C type cast or C++ <code>reinterpret_cast</code>.</p>
<h2>Data and pointers</h2>
<p>A CData object represents a C value in memory. You can always get a pointer to the C value using the <a href="/en/js-ctypes/js-ctypes_reference/CData" title="en/js-ctypes/js-ctypes reference/CData"><code>CData</code></a> object's <a href="/en/js-ctypes/js-ctypes_reference/CData#address()" title="en/js-ctypes/js-ctypes reference/CData#address()"><code>address()</code></a> method.</p>
<h3>Objects can share memory</h3>
<p>It's important to keep in mind that two (or more) <code>CData</code> objects can share the same memory block for their contents. This will happen, for example, when <a href="/en/js-ctypes/Using_js-ctypes/Working_with_data#Type_casting" title="en/js-ctypes/Using js-ctypes/Working with data#Type casting">type casting</a>. This is called <strong>aliasing</strong>. The shared memory can be whole or in part.</p>
<p>For example:</p>
<pre class="brush: js">const Point = new ctypes.StructType("Point", [[ctypes.int32_t, 'x'], [ctypes.int32_t, 'y']]);
const Rect = new ctypes.StructType("Rect", [[Point, 'topLeft'], [Point, 'bottomRight']]);

var r = Rect();
var p = r.topLeft;
r.topLeft.x = 100; 
</pre>
<p>At this point, <code>p</code> is a reference to the <code>topLeft</code> field in the <code>Rect</code> named <code>r</code>. Setting the value of <code>p.x</code> will affect the value of <code>r.topLeft.x</code>, as expected.</p>
<h2>Quirks in equality</h2>
<p>Equality doesn't work the same way in JavaScript as it does in C, which means certain operations might not work the way you expect. In particular, comparing two different objects that are represented under-the-hood as JavaScript objects using the <code>==</code> or <code>===</code> operators will always return <code>false</code>. This affects comparisons of pointers, integers that are the same size as pointers, and 64-bit integers.</p>
<p>You can work around this by serializing these values using the <a href="/en/js-ctypes/js-ctypes_reference/CData#toString()" title="en/js-ctypes/js-ctypes reference/CData#toString()"><code>toString()</code></a> method and comparing using the resulting string.</p>
<p>See <a href="/en/js-ctypes/Using_js-ctypes/Working_with_data#Determining_if_two_pointers_are_equal" title="en/js-ctypes/Using js-ctypes/Working with data#Determining if two pointers are equal">Determining if two pointers are equal</a> for an example of how this works when comparing pointers.</p>
<h3>Example: Checking the value of an integer</h3>
<p>If, for example, you need to see if the value of an integer is 5, you can do so like this:</p>
<pre class="brush: js">var t = ctypes.int32_t(5);

if (t.toString() == "ctypes.int32_t(5)") {
  // it's 5
} 
</pre>
<h2>Working with strings</h2>
<p>C functions expect strings to be arrays of characters, with the end of the string indicated by a null character. JavaScript, on the other hand, uses the <a href="/En/Core_JavaScript_1.5_Reference/Global_Objects/String" title="en/Core JavaScript 1.5 Reference/Global Objects/String"><code>String</code></a> object to represent strings.</p>
<h3>Converting C strings to JavaScript</h3>
<p>The <a href="/en/js-ctypes/js-ctypes_reference/CData" title="en/js-ctypes/js-ctypes reference/CData"><code>CData</code></a> object provides the <a href="/en/js-ctypes/js-ctypes_reference/CData#readString()" title="en/js-ctypes/js-ctypes reference/CData#readString()"><code>readString()</code></a> method, which reads bytes from the specified string and returns a new JavaScript <a href="/En/Core_JavaScript_1.5_Reference/Global_Objects/String" title="en/Core JavaScript 1.5 Reference/Global Objects/String"><code>String</code></a> object representing that string.</p>
<div class="note"><strong>Note:</strong> The source C string is assumed to be UTF-8, and is assumed to be null terminated. If you need to convert a string that doesn't meet these requirements, you'll need to do it yourself.</div>
<p>For example:</p>
<pre>var jsString = timeStr.readString();
</pre>
<h3>Converting JavaScript strings to C</h3>
<p>Happily, converting JavaScript strings to C formatted strings is easy; just create a character array containing the JavaScript string:</p>
<pre>var myUTF8String = ctypes.char.array()("Original string.");
</pre>
<p>This creates a UTF-8 format null-terminated string in the character array named <code>myUTF8String</code>.</p>
<p>If you need a UTF-16 string, you can do this:</p>
<pre>var myUTF16String = ctypes.jschar.array()("Original string.");
</pre>
<div class="note"><strong>Note:</strong> At this time, there's no way to specify a particular encoding; you may only retrieve the string in UTF-8 or UTF-16 as shown above.</div>
<h3>Using strings with C functions</h3>
<p>You don't even need to convert strings when using them as input parameters to C functions. They get converted automatically for you. Just pass in the JavaScript <a href="/en/JavaScript/Reference/Global_Objects/String" title="en/JavaScript/Reference/Global Objects/String"><code>String</code></a> object.</p>
<p>However, when C functions return, they still return a <code>char.ptr</code> or <code>jschar.ptr</code> (that is, a pointer to an 8-bit or 16-bit array of characters). You'll have to convert those yourself, as covered above.</p>
<h2>Working with pointers</h2>
<h3>Reading data referenced by pointer</h3>
<p>This example creates a pointer to an integer, then looks at the pointed-to data using the <code>PointerType</code> object's <code>contents</code> property.</p>
<pre class="deki-transform">var i = ctypes.int32_t(9);    // Create a C integer whose value is 9

var p = i.address();          // Create a pointer to the integer variable i

if (p.contents == 9) {        // Look at the contents of the pointer
  // the value is 9
} else {
  // the value isn't 9
}
</pre>
<h3>Setting data referenced by pointer</h3>
<p>You can also use the <code>contents</code> property to set the value pointed to by a variable.</p>
<pre class="deki-transform">var i = ctypes.int32_t(9);    // Create a C integer variable whose value is 9

var p = i.address();          // Get a pointer to i
p.contents = 12;              // Change the value of i to 12
</pre>
<h3>Checking to see if a pointer is null</h3>
<p>This example demonstrates the use of the {{ manch("isNull") }} method to determine whether or not a pointer is null.</p>
<pre class="deki-transform">var p = someCDataObject.address();

if (p.isNull()) {
  // the pointer is null
} else {
  // the pointer isn't null
}
</pre>
<h3>Determining if two pointers are equal</h3>
<p>Due to quirks in how equality is determined in JavaScript, the best way to see if two pointers are equal is to convert them to strings, then compare the strings.</p>
<pre class="deki-transform">if (i.address().toString() == ctypes.int32_t.ptr(5).toString()) {
  // the integer i's address is 5
}
</pre>
<p>The example above not only compares the addresses, but also the type. So while the above comparison succeeds if the address of <code>i</code> is 5, it also only succeeds if <code>i</code> is in fact of type <code>ctypes.int32_t</code>.</p>
<p>If you don't care about type equality, and simply want to compare two addresses, you can use type casting:</p>
<pre class="deki-transform">if (ctypes.cast(p, ctypes.uintptr_t.value.toString() == "5") {
  // the pointer p's address is 5
}
</pre>
<p>This casts the pointer to a <code>ctypes.uintptr_t</code>, for which the value property returns a <code>ctypes.UInt64</code>. Calling <a href="/en/js-ctypes/js-ctypes_reference/CData#toString%28%29" title="en/js-ctypes/js-ctypes reference/CData#toString()"><code>toString()</code></a> on that returns the pointer as a base 10 integer.</p>
<h2>64-bit integers</h2>
<p>While most numeric types in js-ctypes are represented by standard JavaScript <a href="/cn/Core_JavaScript_1.5_Reference/Global_Objects/Number" title="cn/Core JavaScript 1.5 Reference/Global Objects/Number"><code>Number</code></a> objects, 64-bit integers cannot all be represented accurately by this type. For that reason, 64-bit and pointer-sized C values of numeric types don't get automatically converted to JavaScript numbers. Instead, they're converted to JavaScript objects that you can manipulate using the methods provided by the <a href="/en/js-ctypes/js-ctypes_reference/Int64" title="en/js-ctypes/js-ctypes reference/Int64"><code>Int64</code></a> and <a href="/en/js-ctypes/js-ctypes_reference/UInt64" title="en/js-ctypes/js-ctypes reference/UInt64"><code>UInt64</code></a> objects.</p>
<div class="note"><strong>Note:</strong> These 64-bit integer types are intentionally low on features, so that when JavaScript adds a "big number" type, we can easily upgrade to support that properly.</div>
<h3>Creating 64-bit integer objects</h3>
<p>To create a new 64-bit number object, use the <code>ctypes.Int64.Int64()</code> method to create a new 64-bit signed integer, or the <code>ctypes.UInt64.UInt64()</code> method to create a new unsigned 64-bit integer. For example:</p>
<pre>var num1 = ctypes.UInt64(5000);
var num2 = ctypes.Int64(-42);
</pre>
<p>If you need to create a 64-bit integer whose value can't be represented by a JavaScript <a href="/cn/Core_JavaScript_1.5_Reference/Global_Objects/Number" title="cn/Core JavaScript 1.5 Reference/Global Objects/Number"><code>Number</code></a>, you have two options. You can build the new number using the high and low 32-bit numbers, or you can create it using a string representation.</p>
<h4>Creating a 64-bit value using a string</h4>
<p>You can simply pass a string representation of a 64-bit number into the <a href="/en/js-ctypes/js-ctypes_reference/Int64" title="en/js-ctypes/js-ctypes reference/Int64"><code>Int64</code></a> or <a href="/en/js-ctypes/js-ctypes_reference/UInt64" title="en/js-ctypes/js-ctypes reference/UInt64"><code>UInt64</code></a> constructor, like this:</p>
<pre>var num1 = ctypes.Int64("400000000000");
var num2 = ctypes.Unt64("-0x1234567890ABCDEF");
</pre>
<p>As you can see, you can use this technique with both decimal and hexadecimal source strings.</p>
<h4>Creating a 64-bit value using join()</h4>
<p>The <code>join()</code> method offered by the <a href="/en/js-ctypes/js-ctypes_reference/Int64" title="en/js-ctypes/js-ctypes reference/Int64"><code>Int64</code></a> and <a href="/en/js-ctypes/js-ctypes_reference/UInt64" title="en/js-ctypes/js-ctypes reference/UInt64"><code>UInt64</code></a> objects provides another way to construct 64-bit integers. It accepts as its input parameters the high and low 32-bit values and returns a new 64-bit integer. For example:</p>
<pre>var num = ctypes.UInt64.join(-0x12345678, 0x90ABCDEF);
</pre>
<h4>Performing arithmetic with 64-bit values</h4>
<p>The <a href="/en/js-ctypes/js-ctypes_reference/Int64" title="en/js-ctypes/js-ctypes reference/Int64"><code>Int64</code></a> and <a href="/en/js-ctypes/js-ctypes_reference/UInt64" title="en/js-ctypes/js-ctypes reference/UInt64"><code>UInt64</code></a> objects don't provide any methods for performing arithmetic, which means you'll have to do it yourself by pulling out the high and low 32-bit portions and doing math on them, then joining them back together if necessary to get the complete result. You'll also have to handle carry from the low to high word and back as appropriate.</p>
<p>more to come...</p><h2>See also</h2>
<ul> <li><code><a href="/en/js-ctypes/js-ctypes_reference/CData" title="en/js-ctypes/js-ctypes reference/CData">CData</a><br> </code></li> <li><a href="/en/js-ctypes/js-ctypes_reference/CType" title="en/js-ctypes/js-ctypes reference/CType"><code>CType</code></a></li>
</ul>
Revert to this revision