A high-level framework Apple’s graphics framework called MetalKit makes it simpler to understand Metal. Here is a guide on how to use Apple’s 3D framework.
On Apple devices, Apple offers buttery-smooth 3D rendering at performance levels that are impossible with rival 3D frameworks.
If you have a subscription to and use the Apple Arcade game app, you have probably seen a demonstration of Metal rendering on an iOS or macOS device. Arcade’s quick introductory animation is produced using Metal:
See Today’s AppleInsider TV Videos
Metal was used to produce the Apple Arcade app’s intro.
Apple unveiled MetalKit, a new, more advanced foundation for Metal, during WWDC in 2015. This framework facilitates the use of Metal by offering various higher-level utilities that simplify the creation of 3D applications.
In particular, MetalKit is Apple’s graphics framework offers further Metal APIs in the following fields:
loading textures, model I/O, and view control
loading of textures
Using the MTKTextureLoader class with MetalKit makes loading assets and textures simpler. This class offers a simple method for setting texture parameters as well as loading assets and textures.
Simply put, a mipmap (or MIP map) is a multi-layered image with each layer having a lower resolution than the layer before it. Mipmaps used to speed up the rendering of images and get rid of Moire patterns and other aliasing issues.
Lines or regular pixel patterns, like alternating pixel grids, can occasionally see as disturbing banding or color artifact patterns in computer graphics.
A Moire design.
The complete documentation for MTKTextureLoader could found at Documentation/MetalKit/MTKTextureLoader on Apple’s developer website.
Apple’s developer framework for managing 2D and 3D assets is called Model I/O. Interfaces for fast-loading textures into Metal buffers and utilizing mesh data using containers are included in MetalKit’s Model I/O integration.
Currently, MetalKit has roughly six Model I/O-related classes, most of which deal with meshes. In a moment, we’ll discuss classes and object-oriented programming.
viewing control
Understanding MetalKit: Getting started with Apple’s graphics framework
A little OOP is Apple’s graphics framework.
The classes used in object-oriented programming (OOP) define the objects. A class is the actual implementation of an object in Swift as well as a definition of what an object contains in a source code file.
A class defines methods (functions) that can respond to messages that other objects send to them in order to carry out a certain function. Each method has code that carries out some task.
Also, a class defines variables or attributes that can hold data. Normally, a class’s methods work on the class’s properties in some way. Most methods can read most properties found in the class or one of its superclasses, but not all of them can (parent classes).
This entire process is referred to as object encapsulation. In order to keep everything neat and organized, objects contain both data and methods. Instead of having to keep track of the data individually, it is simpler to move, reference, copy, and use objects with their associated data in one location.
OOP is Apple’s graphics framework
A feature of OOP called inheritance allows new classes to be defined from existing ones. The parent class is referred to as the superclass, while the derived object is referred to as a subclass.
Subclassing allows for the definition of lengthy object chains. Because it makes it so easy to reuse existing code, inheritance is quite effective.
With hardly any additional effort, subclasses effortlessly inherit all the traits and characteristics of their parent classes. Subclasses have the ability to add new methods that only they (or their subclasses) are aware of.
Even better, when you create an instance (one duplicate) of an object in a program, it automatically creates copies of all the objects in its superclass as well.
By simply making one instance of a class, you can add a huge amount of program capability with just one line of code.
Understanding MetalKit:
Just generating an object, providing RAM space for it, and making it accessible to a program is instantiation.
In the case of Objective-C, all of this is often defined in one or two source code files, typically one or two files per class.
Hence, in the example above, an MTKView is a class that is instantiated when it is generated in code and is specified as such (by Apple) or (by you). The outcome is a functioning MTKView object in memory. The MTKView object is destroyed when it is de-allocated, which removes it from memory when it is no longer required.
Many such things can be created, used, managed, and destroyed by the majority of apps.
By encapsulating code and data, the OOP programming paradigm keeps applications more modular and reusable while drastically reducing the amount of code required.
Once you’ve created a class to perform a specific task, you may rapidly construct another app by reusing the class or subclassing it in another program.
MetalKit’s fundamental animation
MTKView has a Core Animation Layer just like many other common iOS or macOS standard views. Apple’s high-performance 2D animation framework is called Core Animation.
A CALayer, a Core Animation layer object that can draw and animate 2D images, is present in the majority of views. Complex animations can be made using the grouping and combining of CALayers.
Metal can render into the CAMetalLayer subclass of CALayer that MTKView has created. To create combined 2D and 3D animations, you can use CAMetalLayer with other CA layers.
Drawing in CALayers is typically faster and more effective than drawing in UIViews in both 2D and 3D applications. You can also make some portions of CA layers transparent by adjusting their opacity or alpha.
MTKView settings
Three drawing modes are supported by MTKView:
Timed Notifications: Explicit With timed drawing, the view is updated at predetermined intervals. When a gaming scene is displayed or drawn at a certain pace expressed in frames per second (fps), the majority of games use this option.
To start and stop the view animation in Timed mode, you can also set or clear the isPaused attribute.
Redrawing occurs in notification mode whenever a portion of the entire display is rendered invalid. This makes it possible to only redraw a section of the view or layer, which speeds up the game’s performance.
Simply send the view object a setNeedsDisplay() message to make it redraw when using notification mode. By sending each of the view’s subviews a setNeedsDisplay() message as well, this compels the view to redraw all of them.
In Explicit drawing, you can redisplay view content by directly sending a draw() message to the view object. Unless you have a unique drawing technique that deviates from the usual view/subview structure, this is generally avoided.
You may also avoid having the top-level view redrawn by sending the setNeedsDisplay() message to the subviews that make up the view you only want to redraw. The performance generally improves as the number of redrawn objects decreases.
You receive an MTLRenderPassDescriptor from the view in the case of an MTKView or a subclass thereof, render into it, and then present the resulting drawable for display in your drawing method.
Any Metal object that has been decoded and is ready to be displayed is drawable.
MTKViewDelegate
A delegate is an object that works on another object’s behalf in Apple’s Swift and Objective-C programming languages.
A delegate object is often declared as one of the attributes of another object, and the delegate declares the methods (functions) it offers.
Delegates are effective because you can alter an object’s behavior by only altering a delegate property. Delegates are also used to extend the functionality of objects without having to build new ones by subclassing existing ones.
The MTKViewDelegate class, which is also covered in Apple’s documentation, is the delegate object that belongs to MTKView. MTKViewDelegate typically reacts to events related to view redraw and resize.
MTKViewDelegate likewise derives from NSObjectProtocol, a standard Objective-C protocol shared by all Apple objects.
Delegates and protocols can be thought of as extra objects and methods that can be “glued” or attached to other objects.
A protocol in Objective-C and Swift is nothing more than a set of extra methods that a class is required to implement. Each class is free to define what each method in a protocol contains.
The MTKViewDelegate is primarily focused on drawing and altering a view’s layout (for example, in response to a device rotation).
By creating many MTKViewDelegate objects, each with a distinct behavior, you might, for instance, modify the drawing or rotation behavior of your MTKView by simply changing the value of its delegate attribute to any of the delegate objects at will and redrawing.
Rendering
Implement the MTKViewDelegate’s methods in your renderer when using MTKView. As a result, your renderer can interact with the MTKView and make changes to the rendering and layout.
Using the currentRenderPassDescriptor attribute of the MTKView, you can learn when a frame should be rendered. This enables interaction with each frame that is going to be presented.
For instance, in Swift,
if onscreenDescriptor = view, let it be.
currentRenderPassDescriptor
This retrieves the onscreenDescriptor variable, which contains the current render pass descriptor for the MTKView.
The drawable must be used to modify the contents of the view after rendering. Use the present(_:) function on the MTLCommandBuffer object to accomplish this, then send the command buffer and commit() message to the GPU for visual display.
In MTKView’s documentation, this procedure is covered in more detail.
SIMD
When managing 3D and 2D objects and running computations on them, as well as matrices, Apple also provides a math-related framework called SIMD that is useful. The majority of these functions are utilized for quick, effective floating point math, which is frequently used in 3D calculations.
When you need to alter 3D objects and object vertices, SIMD can be useful. The SIMD float4x4 data structure, a four-by-four matrix of single-precision floating numbers, is the most popular and practical SIMD data structure.
Shaders
As a Metal game app’s complete code is Apple’s graphics framework outside the purview of this post, we’ll only briefly go through the fundamentals for our one-object example. A single rotating 3D cube is produced by the sample project template provided by Apple.
In addition to a ShaderTypes.h file, which is a header file containing the mesh and vertex information of the shader along with a C struct specifying the projection matrix and model view matrix, Xcode also creates an app delegate file that manages the overall event loop of the app itself.
When drawing, the shader makes use of these.
The “ShaderTypes.h” header file, which is shared by the renderer and the GameViewController.swift file that we’ll access in a second, is imported by the “Shaders. metal” file. With the import preprocessor directive, you can insert header files into other Swift or Objective-C source code files:
“ShaderTypes.h” is imported.
Usually starting with a “#,” preprocessor directives are compiler instructions that execute before the actual compilation.
Using the previous ANSI-C import directive #include, “Shaders. metal” also imports two more files: metal stdlib and simd. h. We won’t get into the specific distinctions between #import and #include here because they are comparable to one another.
You’ll notice this line following that:
utilizing metal nomenclature
By defining them under a namespace, namespaces are a C++ idiom that enables similar or identical parts of code to be declared and isolated. There is a name for each namespace; in this example, metal.
Vertex and ColorInOut structures, as well as a number of functions that define the shaders—in this case, just a vertex shader and fragment shader—are defined in Shaders. metal. A sampler variable in the fragment shader also enables you to define and employ mipmaps if you so choose.
Color data, a Uniforms structure described in SharderTypes.h, and a texture2d declared in the Metal library header “metal texture” are the parameters of the fragment shader method.
Understanding MetalKit: Getting started with Apple’s graphics framework
The projection matrix and the model view matrix are both contained in the Uniforms parameter, as was previously mentioned.
The majority of iOS and macOS programs employ views. Which are common classes that display visual data and UI elements on-screen. Various view subclasses offer various view kinds.
For instance, the base class for views in iOS is UIView, but the basic class for button views is UIButton. You can create extra functionality in iOS or macOS that depends on the standard functionality. Already specified by Apple classes by using object-oriented view classes.
It’s referred to as object inheritance. Consider an object in an app as a collection of code. That includes both the code and the data that the code uses to function. Code can be simply reused and repurposed by additional objects by combining both into objects.
Understanding MetalKit: Getting started with Apple’s graphics framework
A new class called MTKView is Apple’s graphics framework made available, specifically in MetalKit, to developers so they may build fully-featured Metal views in their projects. The view may be drawn and handled optimally by Metal without any additional code thanks to a specific Metal view class.
The developer website’s Documentation/MetalKit/MTKView section contains Apple’s MTKView documentation. To instruct MTKView which device and screen to render Metal objects into, you must first set an MTLD device in one of its settings.
MTK
Moreover, MTKView offers an MTLRenderPassDescriptor upon request, into which you can render your textures. Visit Apple’s developer website and look in the Documentation/Metal/Render Passes section.
