Skip to main content

Integrating Unity and Vuforia with a iOS Swift Project

Tags

Integration

Last week I started building an iOS application in Swift 3.0 and Xcode 8 and was able to get a good portion of the application completed.  Towards the end of last week I became aware of the need to integrate a Unity / Vuforia exported project into my existing iOS Swift project to enable an augmented reality feature in the iOS application I was working on.  At first, I did not think anything of this request because in the past I had only minor difficulties integrating previous augmented reality platforms such as Metaio and Wikitude into my previous Objective-C iOS projects, and I did not think that this one was going to be any different.  One thing I can now say now after successfully completing the integration is that there is a lot more involved than I initially thought there was.  However, in saying that, I think the context of my situation is important to keep in mind.  Typically, I am under the impression that most developers use the Xcode project exported from Unity as the starting point for their application.  That would certainly be an easier approach than integrating a Unity exported Xcode project into an existing Xcode Swift based project.  When you integrate a Unity exported project into your existing project there is a lot of project setup that needs to happen with folder / group structure and altering your project build settings, and these are the settings that I am going to cover in this post, how to successfully integrate an exported Xcode Unity project to an existing Swift based Xcode project. 

Before You Get Started

In the following example I am using Unity 5.4, Vuforia 6, Xcode 8.0 with iOS 10.01 and Swift 3.0.  Although it certainly can be, this example does not assume that your Unity / Vuforia scene needs to be loaded as the RootViewController and can be loaded anywhere needed in your project.  This example is compiled as a combination of my own experience and troubleshooting, multiple documentation resources online, and a big help from a Github repo by Blitz Agency who should be credited for a lot of the code that is used in this post from.  Another great resource to check out if you are developing your project in Objective-C is a this post and video by Frederik Jacques (AKA the-nerd).

Folder Structure

Project and Folder Structure

The folder structure in your project is important.  At the end of the day it can be setup any way you choose, but ultimately you will have to be aware of your project structure because you will need to make the necessary changes in your Build Settings to recognize certain files in your project structure.  In my example that is represented in the screen shot above I am recreating the folder and project structure that comes straight out of the Unity exported project.   Setting up the folder structure like this will ultimately be less work on my end that setting up a custom folder structure.

The folder structure that you setup is very important because it needs to expose all of the .cpp, .m, and .mm files to be compiled and it also needs to expose any binary files to be linked when clang compiles your project.  The tricky part is that the folders that are added to your iOS project from Unity need to be added as Xcode groups but also retain their folder structure in the Xcode project as well.  There are four folders from your exported Unity project that need to be added to your existing Xcode project. The Classes, Libraries, Data, and QCAR folders.  This may differ from project to project and how your Unity export setting are configured, but Classes and Libraries will contain all of your code and binary files and usually Data and QCAR contain data, media, and Unity / Vuforia configuration files.  There are a couple of different methods to adding these folders to your project, but the method and folder structure that worked best for this project was:

  • Create a new Xcode group for Libraries and Classes.
  • Once you have created these groups open the right window pane in Xcode and with the newly created group selected, click the folder icon and create a static folder reference in the root of your project. Do this for both of the Libraries and Classes folders so you have a group for these folders and a static folder in your Xcode project.
  • Now drag in all of the code files from your Unity Classes folder to your new Classes group in Xcode. Make sure you drag in the code files only in the root of this folder and NOT the folders inside the Classes folder yet.
  • Now, just as you did before with Classes and Libraries, create a new Xcode group for the folders inside Classes folder. Currently, the folders are UI, Unity, Native, UnityAds, and PluginBase. After you have a new group, create the static folder references for these groups to in your Xcode project directory inside Classes.
  • After all of your static folder references are created inside Classes in Xcode, drag all of the code for the folders UI, Unity, UnityAds, and PluginBase from Unity into Xcode. As for the Native folder, some people say that the .h assembly files are not needed to keep in the Xcode project and complain about Xcode slowing down if they are left in. I can completely understand these files slowing down Xcode, but I have never been successful removing them from the project. So in this example they have been left in, use your own judgment as to what works for you though. The Native folder does need the .cpp files though.
  • Next drag in all of the code and binaries from the root of the of the Libraries folder in Unity to your Libraries group in Xcode. Again, just the code and not the sub folders.
  • Now, inside your Libraries group, create groups for this folder structure: Plugins > iOS > VuforiaMediaSource > src. After that create static folder references in Xcode using the folder icon in the right pane of Xcode as described above.
  • Drag in the code and binaries from Libraries > Plugins > iOS in your Unity project to your Xcode project.
  • Drag in the code from Libraries > Plugins > iOS > VuforiMediaSource > src in your Unity project to your Xcode project.
  • From your Unity project, drag in the static folder for libil2cpp from the Libraries directory in Unity to your Xcode Libraries group. This folder does not need to be created as a group first.
  • After the code files have been added, navigate out to your Build Phases tab in Xcode and make sure that all of the new .cpp, .m, and .mm files are showing up in your Compile Sources. If they are not, then Xcode is not recognizing them for some reason and you need to go back and make sure that Xcode knows that these files need to be compiled.  
  • From Unity drag in the Data and QCAR folders as static folder references in Xcode and not as groups. Drag these folders into the root of your Xcode project.
Framework Binaries

Frameworks and Binaries

After our folder structure is setup and all of the files have been added to our project we need to ensure that the all of the binaries added to project are showing being recognized so that clang knows to link them at compile time.  Also, we want to ensure that all of the necessary frameworks that Unity and Vuforia use have been added to the project to so there will be no issues at compile time or at run time.  I have added a screen shot of all of the needed binaries and frameworks above, but just to make sure you have these added, navigate to the xcodeproj file > Build Phases > Link Binary With Libraries tab and add any of the need frameworks that are missing from your project.  Here is a list:

  • UIKit
  • MediaPlayer
  • iAD (Optional)
  • CoreVideo
  • Security
  • OpenAL
  • OpenGLES
  • QuartzCore
  • SystemConfiguration
  • Foundation
  • CoreLocation
  • CoreMedia
  • CoreMotion (Optional)
  • AVFoundation
  • CFNetwork
  • CoreGraphics
  • AudioToolbox
  • CoreText
  • libiconv.2.tbd
  • libQCARUnityPlayer.a (Added from Libraries Group)
  • libVuforia.a (Added from Libraries Group)
  • libVuforiaMedia.a (Added from Libraries Group)
  • libiPhone-lib.a (Added from Libraries Group)

Objective-C Bridge and Prefix Headers

If a bridging header has not already been added to your project it is a good time to add one now.  A bridging header will expose needed objects and methods to Swift that are created in Objective-C.  To add a new bridging header to your project click on your xcodeproj file > File > New > File... > Header File.  Name your file ProjectName-Bridging-Header.h, where "ProjectName," is the name of your project.  Set this file a side for a moment and we will come back to it later.

Now, you may have noticed that the folder Classes contained a file called: Prefix.pch.  This is a prefix header and can help Xcode compile faster and reduce the amount of files that are needed to include into each of your project files throughout the codebase.  We will create a custom prefix header also just in case your project gets a bit more complex and there are more source files you need to precompile throughout your project.  To create a custom pch file click on your xcodeproj file > File > New > File... > PCH File.  Name your file PrefixHeader.pch.  We will use the one included from unity as a starting point.  Here is the code you must add to your PrefixHeader.pch file below:

#ifndef PrefixHeader_h
#define PrefixHeader_h
 
#ifdef __OBJC__
    #import <Foundation/Foundation.h>
    #import <UIKit/UIKit.h>
#endif
 
#include "Preprocessor.h"
#include "UnityTrampolineConfigure.h"
#include "UnityInterface.h"
 
#ifndef __OBJC__
#if USE_IL2CPP_PCH
    #include "il2cpp_precompiled_header.h"
#endif
#endif
 
#ifndef TARGET_IPHONE_SIMULATOR
    #define TARGET_IPHONE_SIMULATOR 0
#endif
 
#define printf_console printf
#endif /* PrefixHeader_h */
Linker flags

Build Settings

There are quite a few build settings that need to added or altered.  These are the build settings that worked for my project with Xcode 8.0, iOS 10.0.1, and Swift 3.0.  To alter these build setting, navigate to xcodeproj file > Build Settings >

  • Build Options: Enabled Bitcode: No
  • Linking: Other Linker Flags: -lc++ -weak_framework CoreMotion -weak-ISystem
  • Write Link Map File: Yes
Search Paths
  • Search Paths: Header Search Paths:
    • $(SRCROOT)/Libraries
    • $(SRCROOT)/Classes/Unity
    • $(SRCROOT)/Classes/PluginBase
    • $(SRCROOT)/Classes/UI
    • $(SRCROOT)/Classes
    • $(SRCROOT)
    • $(SRCROOT)/Classes/Native
    • $(SRCROOT)/Libraries/bdwgc/include
    • $(SRCROOT)/Libraries/libil2cpp/include
  • Library Search Paths
    • $(inherited)
    • $(SRCROOT)
    • $(PROJECT_DIR)/Libraries
    • $(PROJECT_DIR)/Libraries/Plugins/iOS
  • Apple LLVM 7.1 - Custom Compiler Flags: Other C Flags: -DINIT_SCRIPTING_BACKEND=1
  • Apple LLVM 7.1 - Custom Compiler Flags: Other C++ Flags: -DINIT_SCRIPTING_BACKEND=1
  • Apple LLVM 7.1 - Language: C Language Dialect: C99[-std=c99]
  • Apple LLVM 7.1 - Language: Precompiled Prefix Header: Yes
  • Apple LLVM 7.1 - Language: Prefix Header: /path/to/custom/prefix/header/added/above/PrefixHeader.pch
  • Apple LLVM 7.1 - Language - C++: C++ Language Dialect: C++11 [-std=c++11]
  • Apple LLVM 7.1 - Language - C++: C++ Standard Library: libc++ (LLVM C++ standard library with C++11 support)
  • Apple LLVM 7.1 - Language - C++: Enabled C++ Runtime Types: No
  • Apple LLVM 7.1 - Language - Modules: Enabled Modules (C and Objective-C): No
  • Apple LLVM 7.1 - Preprocessing: Enabled Foundation Assertions: Yes
  • Apple LLVM 7.1 - Preprocessing: Enabled Strict Checking of objc_msgSend Calls: No
  • Apple LLVM 7.1 - Warnings - All Languages: Empty Loop Bodies: No
  • Apple LLVM 7.1 - Warnings - All Languages: Implicit Boolean Conversions: No
  • Apple LLVM 7.1 - Warnings - All Languages: Implicit Constant Conversions: No
  • Apple LLVM 7.1 - Warnings - All Languages: Implicit Conversion to 32 Bit Type: No
  • Apple LLVM 7.1 - Warnings - All Languages: Implicit Enum Conversions: No
  • Apple LLVM 7.1 - Warnings - All Languages: Implicit Integer to Pointer Conversions: No
  • Apple LLVM 7.1 - Warnings - All Languages: Mismatched Return Type: Yes
  • Apple LLVM 7.1 - Warnings - All Languages: Uninitialized Variables: No
  • Apple LLVM 7.1 - Warnings - All Languages: Unreachable Code: No
  • Apple LLVM 7.1 - Warnings - All Languages: Unused Functions: No
  • Apple LLVM 7.1 - Warnings - Objective C: Direct Usage of 'isa': Yes
  • Apple LLVM 7.1 - Warnings - Objective C: Duplicate Method Definitions: No
  • Apple LLVM 7.1 - Warnings - Objective C: Overriding Deprecated Objective-C Methods: Yes
  • Apple LLVM 7.1 - Warnings - Objective C: Undeclared Selector: No
  • Apple LLVM 7.1 - Warnings - Objective C: Unintentional Root Class: Yes
  • Static Analyzer - Analysis Policy: Misuse of 'nonnull': No
  • Swift Compiler - Code Generation: Objective-C Bridging Header: /path/to/custom/bridging/header.h
  • User-Defined: GCC_THUMB_SUPPORT: No (May have to add by hand)
  • User-Defined: GCC_USE_INDIRECT_FUNCTION_CALLS: No (May have to add by hand)
  • User-Defined: MTL_ENABLE_DEBUG_INFO: No (May have to add by hand)
  • User-Defined: UNITY_RUNTIME_VERSION: 5.3.5f1 (May have to add by hand)
  • User-Defined: UNITY_SCRIPTING_BACKEND: il2cpp (May have to add by hand)

Run Script

From your Unity project, add the MapFileParse and MapFileParse.sh files to the root of your Xcode project.  Under the Build Phases tab of your xcodeproj file add a Run Script to make sure that the following two commands below run every time your project compiles.  This script will use the MapFileParse shell script to add new data to the data directory to be used by Unity and Vuforia.  Make sure that the shell environment is set to /bin/sh.

 

"$PROJECT_DIR/MapFileParser.sh"
rm -rf "$TARGET_BUILD_DIR/$PRODUCT_NAME.app/Data/Raw/QCAR"

Code Integrtion

Now, the moment we have all been waiting for, the code integration!  The first thing that you need to do is locate the main.mm file in the Classes group.  The main.mm should look like this:

#include "RegisterMonoModules.h"
#include "RegisterFeatures.h"
#include <csignal>
 
// Hack to work around iOS SDK 4.3 linker problem
// we need at least one __TEXT, __const section entry in main application .o files
// to get this section emitted at right time and so avoid LC_ENCRYPTION_INFO size miscalculation
static const int constsection = 0;
 
void UnityInitTrampoline();
 
// WARNING: this MUST be c decl (NSString ctor will be called after +load, so we cant really change its value)
const char* AppControllerClassName = "UnityAppController";
 
int main(int argc, char* argv[])
{
    @autoreleasepool
    {
        UnityInitTrampoline();
        UnityParseCommandLine(argc, argv);
 
        RegisterMonoModules();
        NSLog(@"-> registered mono modules %p\n", &constsection);
        RegisterFeatures();
 
        // iOS terminates open sockets when an application enters background mode.
        // The next write to any of such socket causes SIGPIPE signal being raised,
        // even if the request has been done from scripting side. This disables the
        // signal and allows Mono to throw a proper C# exception.
        std::signal(SIGPIPE, SIG_IGN);
 
        UIApplicationMain(argc, argv, nil, [NSString stringWithUTF8String:AppControllerClassName]);
    }
 
    return 0;
}

We need to either delete or rename the main.mm file to anything but main.mm.  I chose to delete the file because it is not needed anymore except for reference if you want to look at it.  Next, create a new Objective-C++ file called UnityHook by clicking on your xcodeproj file > File > New > File... > Cocoa Touch Class and name it UnityHook.  When Xcode creates this file change the name of the .m file to UnityHook.mm so Xcode knows to compile this file as Objective-C++.  This will be the entry point fire up Unity when your Swift project runs.  

Add the following to your UnityHook.h file:

#ifndef UnityUtils_h
#define UnityUtils_h
 
 
void custom_unity_init(int argc, char* argv[]);
 
#endif /* UnityUtils_h */

Add the following to your UnityHook.mm file, if your project does not recognize <csignal> right away, that is OK.  You may have to build your project first for this to happen.

#include "RegisterMonoModules.h"
#include "RegisterFeatures.h"
#include <csignal>
 
// Hack to work around iOS SDK 4.3 linker problem
// we need at least one __TEXT, __const section entry in main application .o files
// to get this section emitted at right time and so avoid LC_ENCRYPTION_INFO size miscalculation
static const int constsection = 0;
 
void UnityInitTrampoline();
 
 
extern "C" void custom_unity_init(int argc, char* argv[]){
    UnityInitTrampoline();
    UnityParseCommandLine(argc, argv);
 
    RegisterMonoModules();
    NSLog(@"-> registered mono modules %p\n", &constsection);
    RegisterFeatures();
 
    // iOS terminates open sockets when an application enters background mode.
    // The next write to any of such socket causes SIGPIPE signal being raised,
    // even if the request has been done from scripting side. This disables the
    // signal and allows Mono to throw a proper C# exception.
    std::signal(SIGPIPE, SIG_IGN);
}

Now, go back to that ProjectName-Bridging-Header.h you created earlier and added the following headers to it to expose these objects to Swift.

#import "UnityHook.h"
#import "UnityAppController.h"
#import "Unity/UnityInterface.h"

Out in the folder Libraries/Plugins/iOS folder there is a file called VuforiaNativeRendererController.mm.  We need to comment out the last line to make the file look like this:

#import "UnityAppController.h"
#import "VuforiaRenderDelegate.h"
 
 
// Unity native rendering callback plugin mechanism is only supported 
// from version 4.5 onwards
#if UNITY_VERSION>434
 
// Exported methods for native rendering callback
extern "C" void VuforiaSetGraphicsDevice(void* device, int deviceType, int eventType);
extern "C" void VuforiaRenderEvent(int marker);
 
#endif
 
// Controller to support native rendering callback
@interface VuforiaNativeRendererController : UnityAppController
{
}
- (void)shouldAttachRenderDelegate;
@end
 
@implementation VuforiaNativeRendererController
 
- (void)shouldAttachRenderDelegate
{
    self.renderDelegate = [[VuforiaRenderDelegate alloc] init];
 
// Unity native rendering callback plugin mechanism is only supported 
// from version 4.5 onwards
#if UNITY_VERSION>434
    UnityRegisterRenderingPlugin(&VuforiaSetGraphicsDevice, &VuforiaRenderEvent);
#endif
}
@end
 
// This line below
//IMPL_APP_CONTROLLER_SUBCLASS(VuforiaNativeRendererController)

Next out in our Classes folder there should be a file named UnityAppController.h.  This is the header file that we are going to work with to interact with Unity in Swift over the bridge.  There are a couple of alteration that we need to make to this file.  Here they are listed below:

  • Add UIKit.  For some reason when this file is loaded over the bridge now if the UIKit objects are recognized.
  • Add the class declaration for UnityViewControllerBasel before the inteface declaration for UnityAppController.  This lets clang know that the type UnityViewControllerBase WILL be loaded.
  • Expose all six of the application state functions as prototypes in the header.
  • Comment out the existing delegate function and create a new one that conforms with Swift.
#pragma once
 
#import <QuartzCore/CADisplayLink.h>
// ******** ADDED ********
#import <UIKit/UIKit.h>
#include "PluginBase/RenderPluginDelegate.h"
 
@class UnityView;
@class DisplayConnection;
@class UnityViewControllerBase;
 
 
@interface UnityAppController : NSObject<UIApplicationDelegate>
{
    UnityView*          _unityView;
    CADisplayLink*      _displayLink;
 
    UIWindow*           _window;
    UIView*             _rootView;
    UIViewController*   _rootController;
    UIView*             _snapshotView;
 
    DisplayConnection*  _mainDisplay;
 
    // we will cache view controllers for fixed orientation
    // auto-rotation view contoller goes to index=0
    UnityViewControllerBase * _viewControllerForOrientation[5];
 
#if !UNITY_TVOS
    UIInterfaceOrientation  _curOrientation;
#endif
 
    id<RenderPluginDelegate>    _renderDelegate;
}
 
// ** ADDED ** All six of these function prototypes were added
- (BOOL)application:(UIApplication*)application didFinishLaunchingWithOptions:(NSDictionary*)launchOptions;
- (void)applicationWillResignActive:(UIApplication*)application;
- (void)applicationDidEnterBackground:(UIApplication*)application;
- (void)applicationWillEnterForeground:(UIApplication*)application;
- (void)applicationDidBecomeActive:(UIApplication*)application;
- (void)applicationWillTerminate:(UIApplication*)application;
 
 
// override it to add your render plugin delegate
- (void)shouldAttachRenderDelegate;
 
// this one is called at the very end of didFinishLaunchingWithOptions:
// after views have been created but before initing engine itself
// override it to register plugins, tweak UI etc
- (void)preStartUnity;
 
// this one is called at first applicationDidBecomeActive
// NB: it will be started with delay 0, so it will run on next run loop iteration
// this is done to make sure that activity indicator animation starts before blocking loading
- (void)startUnity:(UIApplication*)application;
 
// this is a part of UIApplicationDelegate protocol starting with ios5
// setter will be generated empty
@property (retain, nonatomic) UIWindow* window;
 
@property (readonly, copy, nonatomic) UnityView*            unityView;
@property (readonly, copy, nonatomic) CADisplayLink*        unityDisplayLink;
 
@property (readonly, copy, nonatomic) UIView*               rootView;
@property (readonly, copy, nonatomic) UIViewController*     rootViewController;
@property (readonly, copy, nonatomic) DisplayConnection*    mainDisplay;
 
#if !UNITY_TVOS
@property (readonly, nonatomic) UIInterfaceOrientation      interfaceOrientation;
#endif
 
@property (nonatomic, retain) id                            renderDelegate;
@property (nonatomic, copy)                                 void(^quitHandler)();
 
@end
 
// Put this into mm file with your subclass implementation
// pass subclass name to define
 
#define IMPL_APP_CONTROLLER_SUBCLASS(ClassName) \
@interface ClassName(OverrideAppDelegate)       \
{                                               \
}                                               \
+(void)load;                                    \
@end                                            \
@implementation ClassName(OverrideAppDelegate)  \
+(void)load                                     \
{                                               \
    extern const char* AppControllerClassName;  \
    AppControllerClassName = #ClassName;        \
}                                               \
@end                                            \
 
// ******** Commented Out ********
//inline UnityAppController*    GetAppController()
//{
//  return (UnityAppController*)[UIApplication sharedApplication].delegate;
//}
 
// ******** ADDED ********
NS_INLINE UnityAppController* GetAppController()
{
    NSObject<UIApplicationDelegate>* delegate = [UIApplication sharedApplication].delegate;
    UnityAppController* currentUnityController = (UnityAppController *)[delegate valueForKey:@"currentUnityController"];
    return currentUnityController;
}
 
 
#define APP_CONTROLLER_RENDER_PLUGIN_METHOD(method)                         \
do {                                                                        \
    id<RenderPluginDelegate> delegate = GetAppController().renderDelegate;  \
    if([delegate respondsToSelector:@selector(method)])                     \
        [delegate method];                                                  \
} while(0)
 
#define APP_CONTROLLER_RENDER_PLUGIN_METHOD_ARG(method, arg)                \
do {                                                                        \
    id<RenderPluginDelegate> delegate = GetAppController().renderDelegate;  \
    if([delegate respondsToSelector:@selector(method:)])                    \
        [delegate method:arg];                                              \
} while(0)
 
 
 
// these are simple wrappers about ios api, added for convenience
void AppController_SendNotification(NSString* name);
void AppController_SendNotificationWithArg(NSString* name, id arg);
 
void AppController_SendUnityViewControllerNotification(NSString* name);

Next, in the UnityAppController.mm file, there are a few changes needed to be make.  Below are the outlined changes.

  • Add the VuforiaRenderedDelegate.h to the bottom of the import list.
  • Remove the empty shouldAttachRendereDelegate method and replace it with the VudoriaRenderDelegate.
// ******** ADDED ********
// Add to top
#import "VuforiaRenderDelegate.h"
 
 
 
 
/*
   ******** START ADDED ********
   Replace the following method stub:
   - (void)shouldAttachRenderDelegate {}
   with what is below:
 */
 
// Unity native rendering callback plugin mechanism is only supported
// from version 4.5 onwards
#if UNITY_VERSION>434
 
// Exported methods for native rendering callback
extern "C" void VuforiaSetGraphicsDevice(void* device, int deviceType, int eventType);
extern "C" void VuforiaRenderEvent(int marker);
 
#endif
 
- (void)shouldAttachRenderDelegate
{
    self.renderDelegate = [[VuforiaRenderDelegate alloc] init];
 
    // Unity native rendering callback plugin mechanism is only supported
    // from version 4.5 onwards
#if UNITY_VERSION>434
    UnityRegisterRenderingPlugin(&VuforiaSetGraphicsDevice, &VuforiaRenderEvent);
#endif
}
 
/*
   ******** END ADDED ********
 */

Next, it is time to finally get into the Swift part of our project.  Navigate out to the AppDelegate.swift file and remove the @UIApplicationMain decorator above the class declaration for the AppDelegate.

import UIKit
 
//@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {
 
    var window: UIWindow?
}

Next, we need to create a custom main.swift file.  Notice that this is not loaded into the project but can be overridden by added a custom one.  In our main.swift file we will make a call to the UnityHook file to load up Unity before the actual Swift project is fired up.  

import Foundation
// overriding @UIApplicationMain
// http://stackoverflow.com/a/24021180/1060314
 
custom_unity_init(CommandLine.argc,CommandLine.unsafeArgv)
// Entry point to the application
UIApplicationMain(CommandLine.argc,
                  UnsafeMutableRawPointer(CommandLine.unsafeArgv)
                    .bindMemory(
                        to: UnsafeMutablePointer<Int8>.self,
                        capacity: Int(CommandLine.argc)),
                  nil,
                  NSStringFromClass(AppDelegate.self)
)

Next, in the AppDelegate.swift file we can load the UnityAppController object and call the delegate life cycle functions from here when needed.  Alter the AppDelegate.swift file to look like the following below:

import UIKit
 
//@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {
 
    var window: UIWindow?
    var currentUnityController: UnityAppController!
 
    func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool {
        // Override point for customization after application launch.
 
        currentUnityController = UnityAppController()
        currentUnityController.application(application, didFinishLaunchingWithOptions: launchOptions)
        return true
    }
 
    func applicationWillResignActive(_ application: UIApplication) {
        // Sent when the application is about to move from active to inactive state. This can occur for certain types of temporary interruptions (such as an incoming phone call or SMS message) or when the user quits the application and it begins the transition to the background state.
        // Use this method to pause ongoing tasks, disable timers, and throttle down OpenGL ES frame rates. Games should use this method to pause the game.
        currentUnityController.applicationWillResignActive(application)
    }
 
    func applicationDidEnterBackground(_ application: UIApplication) {
        // Use this method to release shared resources, save user data, invalidate timers, and store enough application state information to restore your application to its current state in case it is terminated later.
        // If your application supports background execution, this method is called instead of applicationWillTerminate: when the user quits.
        currentUnityController.applicationDidEnterBackground(application)
    }
 
    func applicationWillEnterForeground(_ application: UIApplication) {
        // Called as part of the transition from the background to the inactive state; here you can undo many of the changes made on entering the background.
        currentUnityController.applicationWillEnterForeground(application)
    }
 
    func applicationDidBecomeActive(_ application: UIApplication) {
        // Restart any tasks that were paused (or not yet started) while the application was inactive. If the application was previously in the background, optionally refresh the user interface.
        currentUnityController.applicationDidBecomeActive(application)
    }
 
    func applicationWillTerminate(_ application: UIApplication) {
        // Called when the application is about to terminate. Save data if appropriate. See also applicationDidEnterBackground:.
        currentUnityController.applicationWillTerminate(application)
    }
}

And so finally, the moment we have all been waiting for!  Now we can load up Unity anywhere in our project by adding the following code below to create a UnityView and load it to the UIViewController.

class MyARBrowserViewController: UIViewController {
 
    //
    // ViewDidLoad
    //
    override func viewDidLoad() {
 
        super.viewDidLoad()
 
        let unityView = UnityGetGLView()
 
        self.view.addSubview(unityView)
        unityView.translatesAutoresizingMaskIntoConstraints = false
    }
}

That is it!  Hopefully you found this post helpful and if you did or your happened to see an issue with this post I would love to hear your feedback.  Leave a comment or start a discussion and I will update the post.  Thank you very much!

Upadted for for iOS 10, Swift 3, and Vuforia 6 on 10/1/2016.

Member for

3 years 9 months
Matt Eaton

Long time mobile team lead with a love for network engineering, security, IoT, oss, writing, wireless, and mobile.  Avid runner and determined health nut living in the greater Chicagoland area.

Comments

Can you please share git link of this project for reference.

I currently do not have a link for this project on Github.  The reason for that is because this is an example of how to integrate a Unity and Vuforia build into an existing Swift project.

Anusha

Mon, 10/03/2016 - 09:48 AM

I have integrated the Unity (5.3.5v) into our Swift (2.3)existing application in xcode 7.3.1v, I can see that its integrated successfully and 3D animations are working fine. But the problem is "No live camera feed".
Camera is opening but showing the blank dark screen but it's able to detect image and loading AR graphics.

Are you able to load the camera or can you try loading the camera?
Thanks in advance.

Hello Anusha, yes, I have seen an issue like this before. However the fix for this issue was that I needed to attach the camera feed to the actual view of the view controller instead of a subview somewhere in the UIViewController's view hierarchy and so I doubt this is the actual problem you are facing.  One thing I would wonder is if this is an OpenGL or Metal issue.  I have seen it where using metal instead of OpenGL in the Unity build settings will also do what you are describing.

Anusha

Mon, 10/03/2016 - 12:09 PM

//This is the code
let unityView = UnityGetGLView()
self.view = unityView
self.navigationController?.pushViewController(self, animated: true)

//But the application crash with the following error
UIViewControllerHierarchyInconsistency: A view can only be associated with at most one view controller at a time! View > is associated with . Clear this association before associating this view with .

Hello Anusha, it looks like the you are getting a UIViewController inconsistency crash.  Whithout seeing all of your ViewController setup it is hard to see what is happening, but a couple things I would start with are pushing your new view controller onto the stack and then once it is on the stack and loaded, then add the unityView as a subview and do not assign it as the view for the view controller.  I would try this and see if it gets you anywhere.

Anusha

Mon, 10/03/2016 - 01:03 PM

Hi Matt,

//This is the code
let unityView = UnityGetGLView()
self.view.addSubview(unityView)
self.navigationController?.pushViewController(self, animated: true)

We tried the above code: From the beginning and faced the "No live camera feed" issue. So i changed the code as stated above.

Is there any alternative way to just get the view from the unity instead of view controller? As I feel the multiple window and view controllers from unity are creating some issues here.

Using the method that you are using does return a single UIView to place wherever you would like.  At the bottom of the UnityAppController.mm file there is a extern C function that matches your method call of UnityGetGLView() and it returns the current unityView being created and assigned the camera view.  If this view is black then I would first check to make sure that the orientations are setup correctly.  Next, I would put a break point in the didFinishLaunchingWithOptions function in UnityAppController.mm because this is where the _unityView is being created.

Lastly, I have seen strange inconsistencies with the first generation iPad Air.  Nothing like the view is totally black, but the view would get choppy when a marker was scanned.

Let me know if this helps.

Hello, Matloob.  No problem at all. Glad you found it helpful.  

Three things that you will want to check out:

  1. That the file you are using to override UIApplication is called main.swift.
  2. Make sure that @UIApplicationMain is commented out in the AppDelegate.swift file.
  3. Make sure you have imported Foundation into the main.swift file.  I did not previously have this in my example.  Sorry about that.

Let me know if that works out.

Thanks for writing this up. Im getting a white screen when I run the project and errors in console.

I notice this output:

Could not find a UIView with CAEAGLLayer or CAMetalLayer layer class that responds to selector renderFrameVuforia
UIView has CAEAGLLayer layer class
UIView does not respond to selector renderFrameVuforia
UIView has CAEAGLLayer layer class
UIView does not respond to selector renderFrameVuforia
Could not find a UIView with CAEAGLLayer or CAMetalLayer layer class that responds to selector renderFrameVuforia

But also this errors:

CameraDevice::getCameraCalibration(): Failed to get camera calibration because the camera is not initialized.

VideoBackgroundConfig with screen size of zero received, skipping config step
Trackable LC-symbol-black lost

And then:

eleting old image data @ 0x0
Created new image data buffer @ 0x10a5d4000 w/ 1382400 bytes

And then lots of null reference exceptions.

Any idea why this is happening? It never asks for the camera permission.

That is strange that is never asks for camera permissions?  Did you add it to the plist and accept it the permission the first time it ran?

Also, what is your default graphics context set to in Unity?  Is it set to Metal or OpenGL?  If it is set to use Metal, try switching it to OpenGL and then see if you still have this issue.

Hey Luke, glad you are up and running now.  Yes, in a storyboard, create a UIView as an IBOutlet and then assign that view to a property in your UIViewController.  Then, as described above in my post, take the GLView, i.e., UnityGetGLView() and add this as a subview of your view in the Storyboard.  Another option would be to add the GLView directly to the self.view of the UIViewController.  Hope that helps.

Thats great thanks. The only thing is that the Unity view always fills the entire screen, is there any way to size it to the size of the UIView that you add it as a subview of?

Hello Luke, no problems I am glad you were able to get th Unity view up and running.  If you cannot directly manipulate the frame of the Unity view to the size that you desire then I would take a look at the UnityAppController.mm to see where the unityView is created to see if you can override that at any point.  It looks like the unityView is created in the didFinishLaunchingWithOptions function with the call to [self createUnityView];.   Take a look at that and see if that gets you anywhere.

Thanks so much for the suggestions. Yeah I did try to change the frame size in UnityAppController but I could never get it to compile without error. I'd be happy with Unity rendering fullscreen if I could render views on top of it, however it always goes to the front, even if I add it to a subview and have another view or element above that view.

Im guessing the unity view is being brought to the front somewhere but I cant find anything in their code. Do you get the same behaviour?

Filippos

Wed, 12/28/2016 - 10:24 PM

Hi Matt, thanks so much for uploading this. In the process of trying to run it after following your instructions, I get "use of undeclared identifier 'UnityParseCommandLine" in the UnityHook.mm file compiler error.

the line it is erroring out on is this:

UnityParseCommandLine(argc, argv);

Any idea on how to fix it? Many thanks

Martin

Mon, 01/23/2017 - 04:37 AM

Hi thanks for the tutorial,

I was wondering if there is any way I can initialize Unity from outside the application didFinsishWithLaunchOptions function in AppDelegate.

Hey thanks for the tutorial,

I'm having issues with the Bridging-header.h file we created. I keep getting this error: '/MyProject-Bridging-Header.h' does not exist. I checked the swift compiler settings in my project and I know that it's correct but for some reason I cannot figure out what is causing the error.

Chris, the first thing I would do is ensure that the Bridging-header.h is named correctly and that through your finder window you are able to locate your briding header also.  Is there two Bridging headers in your project, possibly?

Hey,

Was able to fix the Bridging-header issue and the project compiles/runs, but now when I run the project, the Unity view is launched when the app starts up even though I call the view to run on a button click. Another issue is that the AR object is not being loaded on the image that I focus on with the camera

Adrian

Thu, 02/09/2017 - 08:51 AM

Hi. After a lot of time I'm able to compile the project, but when I show the UnityGetGLView() view it is transparent. It have nothing. No logs, no errors, but nothing in the device. I've checked that the view is added correctly, and also I,ve change the view backgroundColor and it appears in all the screen, but nothing...

i found the same problem.

let unityView = UnityGetGLView()
self.view.addSubview(unityView!)
unityView?.translatesAutoresizingMaskIntoConstraints = false

No error , Build successful , But is nothing unity view on screen.

Victor

Fri, 02/24/2017 - 02:08 PM

I'm getting 20 errors in cstring file. All saying the same "No member name 'X' in the global namespace". I've read people fix it setting "Always Search User Paths" to "No" in the project settings but this didn't work for me. BTW great post!!

Are 20 errors saying all the same "No member named 'X' in the global namespace" where X is:

memcpy, memmove, strcpy, strcat, strncat, memcmp, strcmp, strncmp, strcoll, strxfrm, memchr, strchr, strcspn, strpbrk, strrchr, strspn and strstr.

All in the same file cstring, I'm trying to add unity and vuforia to an old project that I want to add AR. Thanks for the help!!

Veit270

Sat, 03/11/2017 - 01:49 PM

Hey, thanks for this tutorial. I managed to compile my project and run it, but it crashes during launch. Here is the image of the crash. Do you know what could cause it? I think it's something related to global-metadata.dat in Data folder, but I can't figure it out.

Thank you for reporting Veit270, this looks like files were not copied into the correct place or possibly Xcode cannot find a file when starting Unity on launch.  Either way, this looks to be a Unity thing when it tries to load up the environment variables and that looks like the problem based upon the stack track.  Check to make sure all of the files are there in the correct place.

Sridhar Sanapala

Mon, 06/05/2017 - 05:06 AM

Hello,

First of all, thanks for the tutorial. This took some time to understand and implement as explained. And it works.

However if I have the project set to launch using 'Main' storyboard, the below code has no impact.

let unityView = MyUnityViewController()
self.view.addSubview(unityView.rootView)

I've debugged it to the point that removing 'Main' as launch storyboard in project settings does launch unity view or else it does not. I've read your other comment regarding creating a subview for the view in storyboard and adding the unity view as a subview. But it doesn't work.

I've XCode 8.3.2, Unity 5.6.1f1, and the Unity project comprises of Vuforia plugin.

rEdh1t

Sat, 06/24/2017 - 08:30 PM

Hello, thank you for the tutorial.

When i finished this tutorial and tried to build my project, I got a big linker error.

Apple Mach-O Linker (ld) Error Group
: Linker command failed with exit code 1 (use -v to see invocation)

Please help me, to resolve this issue.

xCode 8.3.3, Swift 3.1

No problem!  Glad you enjoyed the tutorial.  I have seen this caused by binaries not being in the correct place in your folder structure or they are not included in your build phase.  Make sure that all of the iOS frameworks have also been included in your build phases.  This would cause a linker error too.

rEdh1t

Sun, 06/25/2017 - 10:17 AM

Thank you for the help. Now everything is okey with the app (build successful).
The problem was the libiPhone-lib.a was an alias to the original lib. I replaced this by the original from the Unity app folder.

Some error in the tutorial:
Error:
Linking: Other Linker Flags: -lc++ -weak_framework CoreMotion -weak-ISystem
Resolved:
Linking: Other Linker Flags: -lc++ -weak_framework CoreMotion -weak-lSystem

In Swift 3.1 and xCode 8.2.1+ we need to add MediaToolbox.framework

Have a nice day. :-)

No problem at all!  Glad to hear I could help and everything is resolved and you are up and running again!

Have a great day.

I've included Unity, Vuforia and Swift then build success. but i try to run swift project and it's error.
"was compiled with optimization - stepping may behave oddly; variables may not be available."

rEdh1t

Fri, 06/30/2017 - 12:41 PM

Hi, I am again. When i try to create an archive to send my app to an other person, i always get linker error.

Apple Mach-O Linker (ld) Error Group
clang: error: linker command failed with exit code 1 (use -v to see invocation)

But when I run the project, build successful and it's working on my device.

rEdh1t

Wed, 07/12/2017 - 02:50 PM

Hi,

I have a problem with this integration. When I finished my project i saw that my project using a lot of energy and 70% of CPU always.

I removed my unity from the swift app and everything was good.

How can I run all the Unity project after I will tap on a button?

Flower

Sat, 07/22/2017 - 02:27 PM

I have the Unity- iOS integration without Vuforia and is working. I was wondering if it is possible when I click a button in my Native Swift iOS app get the score label of a game object "MasterScore" that have .js file as script

static var gsocre: int = 0;
function OnGUI() {
GUI.Label (10, 10, 100, 20), "Score: " + gscore;
}

The question is how can I do in my swift project

var myScore = ""
myScore = "The gscore value fromUnity thats is inside the object "MasterScore""

Thanks in advance, I hope you can tell me a solution for that question.

Short answer to this question is yes, there are many different ways that you can exchange information between Unity and iOS.  Often, it will require an extern "c" function to be exposed in your iOS project that Unity can pass information to when an event is triggered.  In JavaScript there are similar patterns.  

The question you have asked is technically outside the scope of this post, but if you have a look here : http://answers.unity3d.com/questions/387371/passing-a-value-from-unity-to-objective-c.html.  This link should give you the general idea on how to get started.

Keep in mind that the link is sending information from C# to Objective-C, but the idea is very similar with JavaScript and then from Objective-C you would need to expose certain functions to Swift.