Augmented reality / video camera plugin

This plugin was developed by Paradox D&D, specialists for augmented reality. The plugin uses markers in order to identify positions and orientations in a live video image. 20 different marker types are available; the marker images can be found in the folder templates\images\markers. They can be printed and attached to a flat surface for placing 3D objects on them in the video image, as the red car in the screenshot below:

The plugin can also control a video camera and capture images. It offers the following functions:

Video camera functions

vc_init(var number)

Initializes a physical camera with the specified id number. Windows OS enumerates devices starting from id 0, so if the computer has only one camera connected, it will be referenced with number == 0. Returns 0 if the initialization failed.

vc_dialog(var* width, var* height)

Opens a camera configuration dialog. In this panel the user can configure resolution, space color, compression and frames per second. Internally calls vc_getsize to get the selected resolution.

vc_getsize(var* width, var* height)

Gets the current camera resolution and returns it in width and height. The current resolution is defined by the user by means of a configuration dialog called by vc_dialog. Otherwise it reads the last configured resolution.

vc_getbmap(BMAP* frame)

Gets the current frame captured by the camera and writes it in the bitmap frame. The bitmap frame must be created with 3 bytes per pixel, f.i BMAP* frame= bmap_createblack(buffer_width, buffer_height, 24);.

vc_close()

Closes the camera and frees all resources.

Example

#include <ackar.h>

#define CONFIGURE_DIALOG
#define ID_CAMERA 0

BMAP* bmap_video_camera = NULL;				// bmap to render the video camera

PANEL* panel_video_camera =					// panel to visualize the render bmap
{
   layer = -1;
   flags =  SHOW | FILTER;
}

function main()
{
  vec_fill(sky_color,0);					// sky must be transparent for video camera image to be visible
  level_load("");

  if(!vc_init(ID_CAMERA)) error("Camera not connected");

  var buffer_width, buffer_height;
#ifdef CONFIGURE_DIALOG
  vc_dialog(&buffer_width,&buffer_height);
#else
  vc_getsize(&buffer_width,&buffer_height);
#endif

  bmap_video_camera = bmap_createblack(buffer_width, buffer_height, 24);
  panel_video_camera.bmap= bmap_video_camera;

  while(!key_esc)
  {
    vc_getbmap(bmap_video_camera);
    wait(1);
  }
  videocamera_close();
  sys_exit("");
}

//Example where the camera panel fills the screen

....
BMAP* bmap_video_camera = NULL;				// bmap to render the video camera

PANEL* panel_video_camera =					// panel to visualize the render bmap
{
   layer = -1;
   flags =  SHOW | FILTER;
}

var buffer_width, buffer_height;

// adjust the render videocamera plane to the actual screen resolution
void fix_panel_videocamera(int res_cam_x,int res_cam_y) 
{
  if(res_cam_y > 0 && res_cam_x > 0)
  {
    panel_video_camera.scale_x = screen_size.x/res_cam_x;
    panel_video_camera.scale_y = screen_size.y/res_cam_y;
  }
}

function main()
{
   .....
   bmap_video_camera = bmap_createblack(buffer_width, buffer_height, 24);
   panel_video_camera.bmap = bmap_video_camera;	
   fix_panel_videocamera(buffer_width,buffer_height);
   .....	
}

AR tracking functions

ar_init (var* markers, var num_markers, var width, var focal_length)

Initializes the tracking from the image captured by the camera; must be called after the camera was initialized.

Parameters

markers - list of marker id numbers to be detected in the image.
num_markers - number of markers to be tracked.
width - the width of the marker images in mm.
focal_length - the camera's focal length in pixels.

Returns

0 if failed, otherwise nonzero.

ar_update()

Updates the tracking. It should be called after getting a new frame from the camera. When this function is called, the information about detection, location and orientation of the markers is refreshed.

Returns

Number of markers detected on the screen.

ar_detect(var id)

Tells whether a marker is seen by the camera or not. Call ar_update before to ensure detecting the marker.

Parameters

id - marker's id number from the marker catalog.

Returns

0 if marker not detected, nonzero if marker visible.

ar_position(var id, VECTOR* position): VECTOR*

ar_angle (var id, ANGLE* angle): ANGLE*

Gets the current marker's position in 3D coordinates and its orientation in euler angles.

Parameters

id - marker's id number from the marker catalog.
pos - VECTOR* to receive the position, or NULL for returning a temporary vector.
angle - ANGLE* to receive the Euler angle, or NULL for returning a temporary vector containing the angle.

Remarks

Call ar_update before calling these functions to ensure getting the marker's current position and orientation.
The returned marker position is the center of the marker image. As this is normally not the center of the model, the distance of the model center to the marker image center should be added after calling ar_position for having the entity correctly placed on the image surface (see example).

ar_getFov(): var

Returns the defined focal length in terms of pixels.

ar_flipHorizontal(var enable)

Tells the tracking system if the image captured by the camera must be flipped. It should be used for the camera image to work like a mirror. It only affects tracking, the video camera buffer will be unaffected.

Parameters

enable - if nonzero, the tracking system expects that the image is flipped. By default, the system expects the image from the camera is not flipped.

Example

#include <default.c>
#include <ackAR.h>



// flips the camera buffer for mirror mode
function flip_horizontal(PANEL* panel,BOOL enable)
{  
  if(enable) panel.scale_x = -abs(panel.scale_x);
  else panel.scale_x = abs(panel.scale_x);  
  ar_flipHorizontal(enable); 
}

// places the entity in the position corresponding to the marker
function update_tracking(ENTITY* ent,var id)  
{
  set(ent,INVISIBLE);
  if(ar_update())  // update the tracking
  {
    if(ar_detect(id))  // is the marker on screen?
    {
      ar_position(id,ent.x);
      ar_angle(id,ent.pan);
// for placing the entity on the marker surface,
// add the rotated vertical center offset             
      VECTOR* vmin = vec_for_min(NULL,ent);
      vmin.x = 0; vmin.y = 0; vmin.z = -vmin.z;
      vec_rotate(vmin,ent.pan); 
      vec_add(ent.x,vmin);
      reset(ent,INVISIBLE);  // make the entity visible
    }
  }
}

function main()
{
  video_mode = 7;
  d3d_antialias = 1;
  level_load ("");
// sky must be transparent for seeing the video camera panel
  vec_set(sky_color,COLOR_CLEAR);    
// model to be placed on marker 1  
  ENTITY* marker_ent = ent_create("vehicle_1.mdl",NULL,NULL);

// initialize the video camera
  if(!vc_init(0))  error("Video camera not found!");  
  var buffer_width,buffer_height;
  vc_dialog(&buffer_width,&buffer_height);

// cover the screen with a video panel   
  PANEL* video_panel = pan_create(NULL,-1);
  video_panel.bmap = bmap_createblack(buffer_width,buffer_height,24);
  video_panel.scale_x = screen_size.x/buffer_width;
  video_panel.scale_y = screen_size.y/buffer_height;
  flip_horizontal(video_panel,true);
  set(video_panel,SHOW|FILTER);

// initialize tracking for 1 marker of 100 mm
  var markers[1] = { 0 };
  if(!ar_init(markers,1,100,600)) error("Can't initialize tracking!");

  while(!key_esc)
  {
    if(vc_getbmap(video_panel.bmap))  // get frame from video camera
      update_tracking(marker_ent,0);
    wait(1);
  }
// close video camera on exit
  vc_close();
  sys_exit(NULL);
}

Remarks

The engine camera position must be at the origin of the map coordinate system. The orientation of the camera must be 0 degrees in pan and tilt for the correct visualization of 3D models relative to the captured image. It Is recommended to use a panel with negative layer that fills the screen. .
The video camera must be set to noflipped mode (no mirror) by default, i.e. what you see is what video camera sees, for all the markers to be detected.
Markers must be attached to a completely flat surface, must be fully inside the camera image, and must have good black/white contrast for being detected.

► latest version online