Jul 28, 2019

Turning Data into Arrays in Swift 5

An earlier article discussed how to take Data and turn it into an Array of some simple type. Swift 5 deprecated the method withUnsafeBytes that passes an UnsafePointer<T> to the closure. Instead, it includes a version of that method that passes an UnsafeBufferPointer to the closure. This makes the counter-example from that post less attractive, but it doesn't make it any clearer how to get your bytes out.

An UnsafeRawBufferPointer is an untyped pointer, meaning you have to explicitly tell Swift the type of the elements it points to. The way to do that is with bindMemory(to:).

The idea of binding memory doesn't exist in a lot of programming languages. You are telling the compiler, I am going to access this memory as if it were type T. The Swift type system can then verify your access into the memory is correct according to the bound type1.

So our updated plan is:

  1. Access an UnsafeRawBufferPointer with the method withUnsafeBytes
  2. Bind the UnsafeRawBufferPointer to create a typed UnsafeBufferPointer
  3. Use the UnsafeBufferPointer to create an Array.
let data = Data(bytes: [0x01, 0x00, 0x00, 0x00,
                        0x02, 0x00, 0x00, 0x00])

let array = data.withUnsafeBytes {
                    (pointer: UnsafeRawBufferPointer) -> [Int8] in
    let bytes = pointer.bindMemory(to: Int8.self)
    return Array<Int8>(bytes)
}

// array == [1, 0, 0, 0, 2, 0, 0, 0]

The code above interprets the data as 8 1-byte values. It's just as valid to interpret that data as 2 4-byte values, like below.

let data = Data(bytes: [0x01, 0x00, 0x00, 0x00,
                        0x02, 0x00, 0x00, 0x00])

let array = data.withUnsafeBytes {
                    (pointer: UnsafeRawBufferPointer) -> [Int32] in
    let bytes = pointer.bindMemory(to: Int32.self)
    return Array<Int8>(bytes)
}

// array == [1, 2]

As with decoding numbers, if you received data from a Bluetooth device with CoreBluetooth, there is no information to tell you how to decode the object. There's also no guarantee the order of the bytes will match the expected values in the Int32 example above. It's necessary to know ahead of time what the Data object represents.


  1. You could technically bind memory to two different types, and the compiler would let you. Doing so is undefined behavior. 

Oct 26, 2018

Archiving Swift Codable Objects

Swift has support for serializing to JSON and property lists, but sometimes you need an archive, and these simple serialization formats won't do. Archives are better for complex object graphs. They contain type information. They can contain Data without round-tripping through a Base-64 string. And they can resolve multiple references to an object. NSKeyedArchiver and NSKeyedUnarchiver support Codable types. But how do you use them? If you're used to using the convenience class methods, you might try something like this:

class CodableClass: Codable {
    var answer: Int
    var name: String
    public init(answer: Int, name: String) {
        self.answer = answer
        self.name = name
    }
}

let myClass = CodableClass(answer: 42, name: "Arthur Dent")
let data = NSKeyedArchiver.archivedData(withRootObject: myClass)
// "NSInvalidArgumentException", "-[_SwiftValue encodeWithCoder:]: unrecognized selector sent to instance 0x103a6b500"

As commented, this produces a runtime exception because the convenience methods are expecting conformance to NSCoding, and CodableClass doesn't implement NSCoding.

When Codable was released, Foundation added an instance method to NSKeyedArchiver: encodeEncodable(_, forKey:) and one to NSKeyedUnarchiver: decodeDecodable(_, forKey:). To use these, you need to create an instance of NSKeyedArchiver and NSKeyedUnarchiver, respectively.

let myClass = CodableClass(answer: 42, name: "Arthur Dent")
let archiver = NSKeyedArchiver(requiringSecureCoding: true)
do {
    try archiver.encodeEncodable(myClass, forKey: NSKeyedArchiveRootObjectKey)
}
catch {
    fatalError(error.localizedDescription)
}
archiver.finishEncoding()

let unarchiver = NSKeyedUnarchiver(forReadingWith: archiver.encodedData)
unarchiver.decodingFailurePolicy = .setErrorAndReturn
let decoded = unarchiver.decodeDecodable(CodableClass.self, forKey:
                                                    NSKeyedArchiveRootObjectKey)!
unarchiver.finishDecoding()
if let error = unarchiver.error {
    fatalError(error.localizedDescription)
}
else {
    assert(decoded!.name == "Arthur Dent")
    assert(decoded!.answer == 42)
}

As a bonus, since these are Swift methods, they are well-typed. So decodeDecodable(_, forKey:) returns an object of the type you are decoding. In the example above, decoded is of type CodableClass.

The example uses a class, but these methods also work for Swift structs that implement Codable.

Apr 16, 2018

Swift Namespacing, Archiving, and XPC

This is admittedly esteric, but I found no answer when I ran into this problem, so I'm writing it up here. If you're having trouble passing your custom Swift object across an XPC connection, read on.

Swift allows you some control over how your Swift class will be imported into Objective-C. There is an @objc attribute that controls the name that is presented in Objective-C. From Using Swift From Cocoa and Objective-C:

You can use the @objc(name) attribute to change the name of a class, property, method, enumeration type, or enumeration case declaration in your interface as it's exposed to Objective-C code.

But keep reading, and you'll find this nugget about how that attribute affects namespacing:

When you use the @objc(name) attribute on a Swift class, the class is made available in Objective-C without any namespacing. As a result, this attribute can also be useful when migrating an archivable Objective-C class to Swift. Because archived objects store the name of their class in the archive, you should use the @objc(name) attribute to specify the same name as your Objective-C class so that older archives can be unarchived by your new Swift class.

In other words, if your class implements NSCoding or NSSecureCoding, you'll probably want to declare your class like this:

@objc(MyClass) class MyClass { /* ... */ }

even though this might appear redundant. This includes the case of using NSSecureCoding to pass custom objects across an XPC connection. If your custom object is imported from a library, this may not be necessary. But if your custom object is included in two targets (the XPC helper target and the host application target), the object will appear as two different classes because of how Swift namespacing works on the basis of the target.

Mar 05, 2018

Swift Error: Protocol Can only Be Used as a Generic Constraint

Anyone working in Swift for any length of time eventually runs into the following error:

protocol 'XXX' can only be used as a generic constraint because it has Self or associated type requirements

It is probably the single biggest cause of confusion and frustration in Swift. If you're staring at this error and struggling to understand what it means, much less how to address it, don't worry. Pretty much everybody has this reaction. There are many ways you can run into this error, some more surprising than others.

Here are some scenarios that can cause this error, and how to address them all.

You can run into this on your own protocols created with an associated type:

protocol Container {
    associatedtype Element
    var values: [Element] { get set }
}

func takesAContainer(value: Container) { ... }

// error: protocol 'Container' can only be used as a generic constraint because it has Self or associated type requirements

You can run into this on your own protocols defined with Self in a signature:

protocol Combinable {
    func combineWith(other: Self) -> Self
}

func takesACombinable(value: Combinable) { }

//error: protocol 'Combinable' can only be used as a generic constraint because it has Self or associated type requirements

You can run into this from standard library protocols if they are defined with either of the patterns above. Equatable is a common culprit. This tends to catch people off guard, because you probably weren't thinking about Self requirements when you started using Equatable.

func takesAnEquatable(value: Equatable) { ... }

// error: protocol 'Equatable' can only be used as a generic constraint because it has Self or associated type requirements

You can run into this if your protocol inherits from any of the above:

protocol Viewable: Equatable {
    func display()
}

func takesAViewable(viewable: Viewable) { ... }

// error: protocol 'Viewable' can only be used as a generic constraint because it has Self or associated type requirements

Finally, you can run into this if your class composes with any of the above protocols:

class Person {
    var name: String
}

typealias EquatablePerson = Person & Equatable

func takesAnEquatablePerson(value: EquatablePerson) { }

// error: protocol 'EquatablePerson' can only be used as a generic constraint because it has Self or associated type requirements

For many of these errors, the solution is to make the functions generic and restrict the generic parameter to the protocol in question. Here are some examples:

func takesAnEquatable<T: Equatable>(value: T) { }
func takesAContainer<T: Container>(value: T) { }
func takesACombinable<T: Combinable>(value: T) { }
func takesAViewable<T: Viewable>(viewable: T) { }
func takesAnEquatablePerson<T: EquatablePerson>(value: T) { }

All these examples are functions, but you can use the same pattern on a struct or a class:

struct StructWithEquatables<T: Equatable> {
    var values: [T]
    func takesAnEquatable(value: T) { }
}

let swe = StructWithEquatables<Int>(values: [1, 2, 3])

There are still other situations where you will run into this problem. I'll cover them in a future article, along with another strategy on how to deal with it: type erasure. If you're interested, sign up on the sidebar and get notified when it comes out.

Feb 21, 2018

String Slicing in Swift

String slicing is somewhat verbose in Swift. If you're coming from another language, it's not immediately obvious how to do it at all.

Many of us end up cursing the screen after trying to get substrings using patterns from other languages. If you feel the same way, you may have run into something like this:

let str = "πŸ“– Reading πŸ“š is πŸ“™ FUNdamental! 😁"

let substr = str[0]
// error: 'subscript' is unavailable: cannot subscript String with an Int

let substr = str[0..<5]
// error: 'subscript' is unavailable: cannot subscript String with a CountableRange<Int>

Swift Strings use a non-integer index type for reasons of speed and safety:

  • Speed: because an index access will always be a constant-time operation.
  • Safety: because you're guaranteed to get a whole character and not, say, half of a 16-bit character.

I know what you're thinking: that's swell and all, but I just want to grab a range of characters. Here's how you do that.

String has a few methods to get a Substring using integer arguments:

    func prefix(Int) -> Substring
    func suffix(Int) -> Substring
    func dropFirst(Int) -> Substring
    func dropLast(Int) -> Substring

Here are some examples:

let str = "πŸ“– Reading πŸ“š is πŸ“™ FUNdamental! 😁"
str.prefix(1)
// πŸ“–

str.suffix(1)
// 😁

str.dropFirst(2).prefix(9) as Substring
// Reading πŸ“š

str.dropFirst(2).dropLast(2) as Substring
// Reading πŸ“š is πŸ“™ FUNdamental!

There some more methods that return indexes.

    func startIndex -> String.Index
    func index(after: String.Index) -> String.Index
    func index(String.Index, offsetBy: Int) -> String.Index

Here are some examples of those:

let str = "πŸ“– Reading πŸ“š is πŸ“™ FUNdamental! 😁"
let start = str.startIndex
let tenth = str.index(start, offsetBy: 10)
let eleventh = str.index(after: tenth)
let books = str[tenth..<eleventh]
// πŸ“š

Note that subscripting with string indexes is constant time, but creating the index can be linear time. So if you’re advancing through the string, or if you repeatedly access indexed values, it’s better to keep a current index position than to repeatedly advance from the beginning of the string.

In other words, this:

let start = str.startIndex
let tenth = str.index(start, offsetBy: 10)
let sixteenth = str.index(tenth, offsetBy: 6)
for i in 1...1000 {
    let books = str[tenth..<sixteenth]
    // πŸ“š is πŸ“™
}

is better than this:

for i in 1...1000 {
    let start = str.startIndex
    let tenth = str.index(start, offsetBy: 10)
    let sixteenth = str.index(tenth, offsetBy: 6)
    let books = str[tenth..<sixteenth]
    // πŸ“š is πŸ“™
}

or this:

for i in 1...1000 {
    let books = str.dropFirst(10).prefix(6) as Substring
    // πŸ“š is πŸ“™
}

Note that these are by no means a complete set of methods. If you come across any other useful methods or patterns of accessing substrings, let me know.

Feb 19, 2018

Interpreting Data using Swift Generics

I’ve written previous posts on how to interpret Data as a simple scalar, or as an array of scalars. Now I want to refine that with generics.

Here’s an example of the code I want to use.

let data:Data = characteristic.value //get a data object from the CBCharacteristic
let temperature: Int16 = data.decode() //Custom method to decode into Int16

My previous solution just called a method that required specifying the pointer type for the closure parameter. In order to make this a method on Data that might return any type, it has to be generic. Here’s an implementation for β€œtrivial” types.

extension Data {
    func decode<T>() -> T? {
        guard self.count == MemoryLayout<T>.size else {
            return nil
        }
        return self.withUnsafeBytes { (ptr: UnsafePointer<T>) -> T in
            return ptr.pointee
        }
    }
}

As long as the size of the Data matches the size of type T, it will interpret it as T. I could contraint T to be a FixedWidthInteger or some known trivial type.

What about arrays of integers? We can use a generic constraint on T to make sure we are only dealing with an array of FixedWidthIntegers, which covers all our common cases.

extension Data {
    func decode<T: FixedWidthInteger>() -> [T]? {
        let byteCount = self.count
        let stride = MemoryLayout<T>.stride
        guard byteCount % stride == 0  else {
            return nil
        }
        return self.withUnsafeBytes({ (ptr: UnsafePointer<T>) -> [T] in
            let count =  byteCount / stride
            let buffer = UnsafeBufferPointer(start: ptr, count: count)
            return Array<T>(buffer)
        })
    }
}

This time I check to make sure the bytes can be evenly divided by elements of size T. Here are some uses. Note that you need to provide type annotations to properly interpret the generic type.

let fourBytes = Data([0x01, 0x02, 0x03, 0x04])
let eightBytes = Data([0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08 ])

let fourByteInt8Array: [Int8]? = fourBytes.decode()
// [1, 2, 3, 4]
let fourByteInt32: Int32? = fourBytes.decode()
// 67305985
let eightByteInt8Array: [Int8]? = eightBytes.decode()
// [1, 2, 3, 4, 5, 6, 7, 8]
let eightByteInt32Array: [Int32]? = eightBytes.decode()
// [67305985, 134678021]

Dec 18, 2017

Turning Numbers into Data in Swift

I've written articles1 on getting raw data into Swift types. This can happen when reading data from a device using Core Bluetooth. Writing data to a Bluetooth Device requires going the other direction and turning your numbers or arrays into raw Data. Luckily, going to Data is a bit easier.

The Data() constructor takes a Sequence. A type of UnsafeRawBufferPointer provides a Sequence interface to iterate over the bytes as UInt8. Creating a Data representation is a matter of getting an UnsafeRawBufferPointer.2

If you have a single scalar, there is a global function withUnsafeBytes(of: body:) that presents a closure

var i: Int32 = 500 // must be a var to pass as inout parameter
let data = withUnsafeBytes(of: &i) { (ptr) -> Data in
    // ptr is an UnsafeRawBufferPointer, which is a Collection
    return Data(ptr)
}

Arrays have an instance method to access the bytes:

let array: [UInt32] = [1, 2, 3]
let data = array.withUnsafeBytes { (ptr) -> Data in
    return Data(ptr)
}

Make sure to use the correct width (8 bit, 16 bit, 32 bit) based on what the device is expecting on the other side. Using Int will usually get you 64-bit Integers, but not always. It's best to be explicit when a specific width is necessary.

As always when dealing with network communication, you might need to switch endianness for the device on the other side.

let myArray : [UInt32] = [1, 2, 3]
let bigEndianArray = myArray.map() { $0.bigEndian }
let bigEndianData = bigEndianArray.withUnsafeBytes { (ptr) in
    ptr.map
    return Data(ptr)
}

  1. See: Turning Data Into Arrays In Swift and Turning Data Into Numbers in Swift 

  2. There are other paths to the same route, like getting an UnsafeBufferPointer<T> to pass to Data(buffer:)

Aug 07, 2017

Turning Data into Arrays in Swift

Update: I wrote this article for Swift 4. There is a newer version of this article for Swift 5.

Last time I talked about turning a Data object into a single scalar. Now I'm going to turn a Data object into an array of scalars.

Based on the last post, it's tempting to try something like this:

let data = Data(bytes: [0x01, 0x00, 0x00, 0x00,
                        0x02, 0x00, 0x00, 0x00])

let array = data.withUnsafeBytes {
                    (pointer: UnsafePointer<[Int8]>) -> [Int8] in
    return pointer.pointee // EXC_BAD_ACCESS
}

For that to work, the sequence of bytes that start at pointer must match the in memory representation of a Swift array containing those eight Int8 values. C arrays work like that. Swift arrays don't. Swift arrays are more complex than a sequence of bytes. If you tried the code above, you'd get a runtime crash.

Swift uses UnsafeBufferPointer to point to some sequence of bytes in memory. You can create an UnsafeBufferPointer from an UnsafePointer. Array has an initializer that takes such a buffer pointer a Sequence, and an UnsafeBufferPointer conforms to Sequence. Here's how to turn that data into an array of Int8 values.

So the plan is:

  1. Access an UnsafePointer with the method withUnsafeBytes
  2. Use the UnsafePointer to create an UnsafeBufferPointer
  3. Use the UnsafeBufferPointer to create an Array.

The type of the Array elements will be the type of the UnsafePointer in the closure given to withUnsafeBytes. Here's one way to do this to create an [Int8].

let data = Data(bytes: [0x01, 0x00, 0x00, 0x00,
                        0x02, 0x00, 0x00, 0x00])

let array = data.withUnsafeBytes {
                    (pointer: UnsafePointer<Int8>) -> [Int8] in
    let buffer = UnsafeBufferPointer(start: pointer,
                                     count: data.count)
    return Array<Int8>(buffer)
}

// array == [1, 0, 0, 0, 2, 0, 0, 0]

The code above interprets the data as 8 1-byte values. It's just as valid to interpret that data as 2 4-byte values, like below.

let data = Data(bytes: [0x01, 0x00, 0x00, 0x00,
                        0x02, 0x00, 0x00, 0x00])

let array = data.withUnsafeBytes {
                    (pointer: UnsafePointer<Int32>) -> [Int32] in
    let buffer = UnsafeBufferPointer(start: pointer,
                                     count: data.count / 4)
    return Array<Int32>(buffer)
}

// array == [1, 2]

As with [decoding numbers][1], ff you received data from a Bluetooth device with CoreBluetooth, there is no information to tell you how to decode the object. There's also no guarantee the order of the bytes will match the expected values in the Int32 example above. It's necessary to know ahead of time what the Data object represents.

Mar 20, 2017

Turning Data into Numbers in Swift

When you read from a Bluetooth Device, Core Bluetooth returns a Data object with the contents. A Data object is basically a container for raw bytes without a type. How do you get that data into a proper Swift type like an Int or a Float? Swift's type system is strict enough that you can't force a typecast to an unrelated type. You can't cast the bytes contained in a Data object to an Int. There is no Int constructor that takes a Data or raw bytes. The method withUnsafeBytes is the method you need for all your data conversion needs.

Before you start, you have to know what the data actually is. There is no way of knowing how to interpret the data without knowledge of the device that's sending you that data. Hopefully there is some documentation to refer to.

The method withUnsafeBytes take a closure, and it passes a pointer into that closure. The type of the pointer is a generic UnsafePointer<T>, and you define what the type of data the pointer contains. If you know you were sent an a 32-bit integer, set your closure parameter as UnsafePointer<Int32>.

The body of that closure is your opportunity to move the data into your desired type. In this case my desired type is an Int32. I can access the Int32 value of the pointer through the pointee property.

The return type of the closure is also generic, and the return type I declare is the return type of the function itself. If I return an Int32 from the closure, the method withUnsafeBytes returns an Int32.

Here's how it looks. Pretty short and easy once you have all the pieces straight.

let data:Data = characteristic.value //get a data object from the CBCharacteristic
let number = data.withUnsafeBytes {
                            (pointer: UnsafePointer<Int32>) -> Int32 in
    return pointer.pointee
}
//number is an Int32

Why an Int32 instead of an Int? Because the size of an Int can change depending on what platform you're running on. The remote device is always going to send you a fixed-length number. If you know it's a 32-bit value, you should force the type to match that exactly.

It's also a good idea to make sure the length of the data matches the size of the type. That is, make sure the data has 32 bytes of data to read before reading the pointee property. Here's one way to do that.

let data:Data = characteristic.value //get a data object from the CBCharacteristic
let number: Int32 = data.withUnsafeBytes {
                            (pointer: UnsafePointer<Int32>) -> Int32? in
    if MemoryLayout<Int32>.size != data.count { return nil }
    return pointer.pointee
}

Take the unsafe in this method name seriously. It's possible to declare a closure that reads more bytes than you have. The compiler won't stop me from doing this, for example:

let data = Data(bytes: [0x00]) // one byte of data
let number: Int32 = data.withUnsafeBytes {
                            (pointer: UnsafePointer<Int32>) -> Int32 in
    return pointer.pointee // reading four bytes of data
}

The value of the number in that case is undefined, and will likely be different on successive runs of the program. Or maybe you'll get a crash.

One final warning. I mentioned that you can't know the data type without some documentation. Even if you know that the number is a 32-bit integer, you also have to know what the byte order of the number is. Your Bluetooth device could be sending a big-endian number to your little-endian iPhone. If you need to change byte order, Core Foundation has utilities like CFSwapInt32BigToHost().

Mar 13, 2017

Converting a Core Bluetooth CBUUID to a Foundation UUID

Here's a short and sweet extension that converts a CoreBluetooth CBUUID to a Foundation UUID. It's interesting mainly because of the time it took me to figure out how to use the raw data.1 I had to get familiar with pointer functions in Swift 3.

UUID can be initialzed with a uuid_t. I just had to find a way to convert a CBUUID to a uuid_t. In the end it turned out to be pretty short and simple. Here are the steps:

  1. CBUUID has a data property that returns a Data object.
  2. Data has a method withUnsafeBytes which gives me an UnsafePointer<T> to work on in the context of a closure. I can designate it to be UnsafePointer<uuid_t>.
  3. UUID has a constructor that takes a uuid_t. I can use that to construct a new UUID which is returned from the closure.

Here's my code:

extension CBUUID {
    var UUIDValue: UUID? {
        get {
            if self.data.count != MemoryLayout<uuid_t>.size { return nil }
            return self.data.withUnsafeBytes {
                            (pointer: UnsafePointer<uuid_t>) -> UUID in
                return UUID(uuid:pointer.pointee)
            }
        }
    }
}

Update: Swift 5

The Data method withUnsafeBytes no longer passes an UnsafePointer<T> to the closure. Now the closure receives an UnsafeRawBufferPointer.

extension CBUUID {
    var UUIDValue: UUID? {
        get {
            if self.data.count != MemoryLayout<uuid_t>.size { return nil }
            return self.data.withUnsafeBytes {
                            (buffer: UnsafeRawBufferPointer) -> UUID in
                let pointer = buffer.bindMemory(to: uuid_t.self)
                return UUID(uuid:pointer.baseAddress!.pointee)
            }
        }
    }
}

A few notes on what I learned from this:

  • Swift sees uuid_t as a 16 element tuple of UInt8s. Because of that, my first attempt used an UnsafePointer<UInt8>.
  • NSUUID has a constructor not available in UUID: init(uuidBytes bytes: UnsafePointer<UInt8>!) For no particularly good reason, I wanted a path to directly construct a UUID.
  • I can manually make a tuple by indexing through an UnsafeBufferPointer<UInt8>, like (buffer[0], buffer[1], ...). That felt inelegant.
  • I can point directly to a uuid_t because the data is layout compatible with the data provided by CBUUID. That is, it has the same elements, the same length, in the same order.
  • CBUUID supports some UUIDs which are shorter than 16 bytes. Usually these are defined in the Bluetooth standard. These are not layout compatible with uuid_t. Instead of converting these to their 16 byte equivalents, I cop out and return nil.

  1. I could have gone from a CBUUIDStringUUID, but that seems unnecessary. A uuid is essentially an array of 16 bytes, and I should be able to change the representation of those bytes without a back-and-forth converstion to String. 

Next → Page 1 of 2