I'm looking for a solution to take a picture or point the camera at a piece of clothing and match that image with an image the user has stored in my app.
I'm storing the data in a Core Data database as a Binary Data object. Since the user also takes the pictures they store in the database I think I cannot use pre-trained Core ML models.
I would like the matching to be done on device if possible instead of going to an external service. That will probably describe the item based on what the AI sees, but then I cannot match the item with the stored images in the app.
Does anyone know if this is possible with frameworks as Vision or VisionKit?
Machine Learning
RSS for tagCreate intelligent features and enable new experiences for your apps by leveraging powerful on-device machine learning.
Posts under Machine Learning tag
76 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
Hi all,
I'm trying to build a scam detection in Message Filter powered by CoreML. I find the predictions of ML reliable and the solution for text frauds and scams are sorely needed.
I was able to create a trained MLModel and deploy it in the app. It works on my container app, but when I try to use and initialise the model in the Message Filter extension, I get an error;
initialization of text classifier model with model data failed
I have tried putting the model in the container app, extension, even made a shared framework for container and extension but to no avail. Every time I invoke the codes to init my model from the extension, I am met with the same error.
Here's my code for initializing the model
do {
let model = try Ace_v24_6(configuration: .init())
let output = try model.prediction(text: text)
guard !output.label.isEmpty else {
return nil
}
return MessagePrediction(rawValue: output.label)
} catch {
return nil
}
My question is: Is it impossible to use CoreML in MessageFilters?
Cheers
Hey, I have been trying out the Xcode 16 beta's code completion for the last couple of days. I went do disable it through the settings, in the components section. I did this to go along with a tutorial and I didn't want it to help me out.
But now that I want it back. I can't find a way to enable it again.
I tried reinstalling both the beta and regular xcode, but it didn't show up again.
Wanted to ask if someone knows how to get this back. Thanks!
I created a model that classifies certain objects using yolov8. I noticed that the model is not working properly in my application. While the model works fine in Xcode preview, in the application it either returns the same result with 99% accuracy for each classification or does not provide any result.
In Preview it looks like this:
Predictions:
extension CameraVC : AVCapturePhotoCaptureDelegate {
func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: (any Error)?) {
guard let data = photo.fileDataRepresentation() else {
return
}
guard let image = UIImage(data: data) else {
return
}
guard let cgImage = image.cgImage else {
fatalError("Unable to create CIImage")
}
let handler = VNImageRequestHandler(cgImage: cgImage,orientation: CGImagePropertyOrientation(image.imageOrientation))
DispatchQueue.global(qos: .userInitiated).async {
do {
try handler.perform([self.viewModel.detectionRequest])
} catch {
fatalError("Failed to perform detection: \(error)")
}
}
lazy var detectionRequest: VNCoreMLRequest = {
do {
let model = try VNCoreMLModel(for: bestv720().model)
let request = VNCoreMLRequest(model: model) { [weak self] request, error in
self?.processDetections(for: request, error: error)
}
request.imageCropAndScaleOption = .centerCrop
return request
} catch {
fatalError("Failed to load Vision ML model: \(error)")
}
}()
This is where i print recognized objects:
func processDetections(for request: VNRequest, error: Error?) {
DispatchQueue.main.async {
guard let results = request.results as? [VNRecognizedObjectObservation] else {
return
}
var label = ""
var all_results = []
var all_confidence = []
var true_results = []
var true_confidence = []
for result in results {
for i in 0...results.count{
all_results.append(result.labels[i].identifier)
all_confidence.append(result.labels[i].confidence)
for confidence in all_confidence {
if confidence as! Float > 0.7 {
true_results.append(result.labels[i].identifier)
true_confidence.append(confidence)
}
}
}
label = result.labels[0].identifier
}
print("True Results " , true_results)
print("True Confidence ", true_confidence)
self.output?.updateView(label:label)
}
}
I converted the model like this:
from ultralytics import YOLO
model = YOLO(model_path)
model.export(format='coreml', nms=True, imgsz=[720,1280])
APP Intent is generated through the "Extract app intents metadata" swift compiled step, thus make it difficult to delete. Is there any API can delete an APP Intent at runtime instead of wait for the version release ?
I am currently working on a 2D pose estimator. I developed a PyTorch vision transformer based model with 17 joints in COCO format for the same and then converted it to CoreML using CoreML tools version 6.2.
The model was trained on a custom dataset. However, upon running the converted model on iOS, I observed a significant drop in accuracy. You can see it in this video (https://youtu.be/EfGFrOZQGtU) that demonstrates the outputs of the PyTorch model (on the left) and the CoreML model (on the right).
Could you please confirm if this drop in accuracy is expected and suggest any possible solutions to address this issue? Please note that all preprocessing and post-processing techniques remain consistent between the models.
P.S. While converting I also got the following warning. :
TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if x.numel() == 0 and obsolete_torch_version(TORCH_VERSION, (1, 4)):
P.P.S. When we initialize the CoreML model on iOS 17.0, we get this error:
Validation failure: Invalid Pool kernel width (13), must be [1-8] or 20.
Validation failure: Invalid Pool kernel width (9), must be [1-8] or 20.
Validation failure: Invalid Pool kernel width (13), must be [1-8] or 20.
Validation failure: Invalid Pool kernel width (9), must be [1-8] or 20.
Validation failure: Invalid Pool kernel width (13), must be [1-8] or 20.
This neural network model does not have a parameter for requested key 'precisionRecallCurves'. Note: only updatable neural network models can provide parameter values and these values are only accessible in the context of an MLUpdateTask completion or progress handler.
I made a model using pytorch and then converted it into a mlmodel file. Next I tried and downloaded (https://developer.apple.com/documentation/vision/recognizing_objects_in_live_capture) which worked! But when I changed the model to my model that I made, the camera worked, but no predictions where shown please
h
elp!
Hi, this is the 3rd time I'm trying to post this on the forum, apple moderators ignoring it.
I'm a deep learning expert with a specialization of image processing.
I want to know why I have hundreds of AI models on my Mac that are indexing everything on my computer while it is idle, using programs like neuralhash that I can't find any information about.
I can understand if they are being used to enhance the user experience on Spotlight, Siri, Photos, and other applications, but I couldn't find the necessary information on the web.
Usually, (spyware) software like this uses them to classify files in an X/Y coordinate system. This feels like a more advanced version of stuxnet.
find / -type f -name "*.weights" > ai_models.txt
find / -type f -name "*labels*.txt" > ai_model_labels.txt
Some of the classes from the files;
file_name: SCL_v0.3.1_9c7zcipfrc_558001-labels-v3.txt
document_boarding_pass
document_check_or_checkbook
document_currency_or_bill
document_driving_license
document_office_badge
document_passport
document_receipt
document_social_security_number
hier_curation
hier_document
hier_negative
curation_meme
file_name: SceneNet5_detection_labels-v8d.txt
CVML_UNKNOWN_999999
aircraft
automobile
bicycle
bird
bottle
bus
canine
consumer_electronics
feline
fruit
furniture
headgear
kite
fish
computer_monitor
motorcycle
musical_instrument
document
people
food
sign
watersport
train
ungulates
watercraft
flower
appliance
sports_equipment
tool
I'm trying to create an app that uses artificial intelligence technology.
One of the models provided on this website(https://developer.apple.com/machine-learning/models/) will be used.
Are there any copyright or legal issues if I create an app using the model provided by this website and distribute it to the App Store?
I'm trying to create an app that uses artificial intelligence technology.
One of the models provided on this website(https://developer.apple.com/machine-learning/models/) will be used.
Are there any copyright or legal issues if I create an app using the model provided by this website and distribute it to the App Store?
Hi, i have been noticing some strange issues with using CoreML models in my app. I am using the Whisper.cpp implementation which has a coreML option. This speeds up the transcribing vs Metal.
However every time i use it, the app size inside iphone settings -> General -> Storage increases - specifically the "documents and data" part, the bundle size stays consistent. The Size of the app seems to increase by the same size of the coreml model, and after a few reloads it can increase to over 3-4gb!
I thought that maybe the coreml model (which is in the bundle) is being saved to file - but i can't see where, i have tried to use instruments and xcode plus lots of printing out of cache and temp directory etc, deleting the caches etc.. but no effect.
I have downloaded the container of the iphone from xcode and inspected it, there are some files stored inthe cache but only a few kbs, and even though the value in the settings-> storage shows a few gb, the container is only a few mb.
Please can someone help or give me some guidance on what to do to figure out why the documents and data is increasing? where could this folder be pointing to that is not in the xcode downloaded container??
This is the repo i am using https://github.com/ggerganov/whisper.cpp the swiftui app and objective-C app both do the same thing i am witnessing when using coreml.
Thanks in advance for any help, i am totally baffled by this behaviour
Hi, I try to create some machine learning model for each stock in S&P500 index. When creating the model(Boosted tree model) I try to make it more successfully by doing hyper parameters using GridSearchCV. It takes so long to create one model so I don't want to think of creating all stocks models. I tried to work with CreateML and swift but it looks like it takes longer to run than sklearn on python.
My question is how can I make the process faster? is there any hyper parameters on CreateML on swift (I couldn't find it at docs) and how can I run this code on my GPU? (should be much faster).
This model run coreml result is not right, the precision is completely wrong, I posted a PhotoDepthAnythingConv.onnx model: https://github.com/MoonCodeMaster/CoremlErrorModel/tree/main/DepthAnything
Hello,
I have created a Neural Network → K Nearest Neighbors Classifier with python.
# followed by k-Nearest Neighbors for classification.
import coremltools
import coremltools.proto.FeatureTypes_pb2 as ft
from coremltools.models.nearest_neighbors import KNearestNeighborsClassifierBuilder
import copy
# Take the SqueezeNet feature extractor from the Turi Create model.
base_model = coremltools.models.MLModel("SqueezeNet.mlmodel")
base_spec = base_model._spec
layers = copy.deepcopy(base_spec.neuralNetworkClassifier.layers)
# Delete the softmax and innerProduct layers. The new last layer is
# a "flatten" layer that outputs a 1000-element vector.
del layers[-1]
del layers[-1]
preprocessing = base_spec.neuralNetworkClassifier.preprocessing
# The Turi Create model is a classifier, which is treated as a special
# model type in Core ML. But we need a general-purpose neural network.
del base_spec.neuralNetworkClassifier.layers[:]
base_spec.neuralNetwork.layers.extend(layers)
# Also copy over the image preprocessing options.
base_spec.neuralNetwork.preprocessing.extend(preprocessing)
# Remove other classifier stuff.
base_spec.description.ClearField("metadata")
base_spec.description.ClearField("predictedFeatureName")
base_spec.description.ClearField("predictedProbabilitiesName")
# Remove the old classifier outputs.
del base_spec.description.output[:]
# Add a new output for the feature vector.
output = base_spec.description.output.add()
output.name = "features"
output.type.multiArrayType.shape.append(1000)
output.type.multiArrayType.dataType = ft.ArrayFeatureType.FLOAT32
# Connect the last layer to this new output.
base_spec.neuralNetwork.layers[-1].output[0] = "features"
# Create the k-NN model.
knn_builder = KNearestNeighborsClassifierBuilder(input_name="features",
output_name="label",
number_of_dimensions=1000,
default_class_label="???",
number_of_neighbors=3,
weighting_scheme="inverse_distance",
index_type="linear")
knn_spec = knn_builder.spec
knn_spec.description.input[0].shortDescription = "Input vector"
knn_spec.description.output[0].shortDescription = "Predicted label"
knn_spec.description.output[1].shortDescription = "Probabilities for each possible label"
knn_builder.set_number_of_neighbors_with_bounds(3, allowed_range=(1, 10))
# Use the same name as in the neural network models, so that we
# can use the same code for evaluating both types of model.
knn_spec.description.predictedProbabilitiesName = "labelProbability"
knn_spec.description.output[1].name = knn_spec.description.predictedProbabilitiesName
# Put it all together into a pipeline.
pipeline_spec = coremltools.proto.Model_pb2.Model()
pipeline_spec.specificationVersion = coremltools._MINIMUM_UPDATABLE_SPEC_VERSION
pipeline_spec.isUpdatable = True
pipeline_spec.description.input.extend(base_spec.description.input[:])
pipeline_spec.description.output.extend(knn_spec.description.output[:])
pipeline_spec.description.predictedFeatureName = knn_spec.description.predictedFeatureName
pipeline_spec.description.predictedProbabilitiesName = knn_spec.description.predictedProbabilitiesName
# Add inputs for training.
pipeline_spec.description.trainingInput.extend([base_spec.description.input[0]])
pipeline_spec.description.trainingInput[0].shortDescription = "Example image"
pipeline_spec.description.trainingInput.extend([knn_spec.description.trainingInput[1]])
pipeline_spec.description.trainingInput[1].shortDescription = "True label"
pipeline_spec.pipelineClassifier.pipeline.models.add().CopyFrom(base_spec)
pipeline_spec.pipelineClassifier.pipeline.models.add().CopyFrom(knn_spec)
pipeline_spec.pipelineClassifier.pipeline.names.extend(["FeatureExtractor", "kNNClassifier"])
coremltools.utils.save_spec(pipeline_spec, "../Models/FaceDetection.mlmodel")
it is from the following tutorial: https://machinethink.net/blog/coreml-training-part3/
It Works and I were am to include it into my project:
I want to train the model via the MLUpdateTask:
ar batchInputs: [MLFeatureProvider] = []
let imageconstraint = (model.model.modelDescription.inputDescriptionsByName["image"]?.imageConstraint)
let imageOptions: [MLFeatureValue.ImageOption: Any] = [
.cropAndScale: VNImageCropAndScaleOption.scaleFill.rawValue]
var featureProviders = [MLFeatureProvider]()
//URLS where images are stored
let trainingData = ImageManager.getImagesAndLabel()
for data in trainingData{
let label = data.key
for imgURL in data.value{
let featureValue = try MLFeatureValue(imageAt: imgURL, constraint: imageconstraint!, options: imageOptions)
if let pixelBuffer = featureValue.imageBufferValue{
let featureProvider = FaceDetectionTrainingInput(image: pixelBuffer, label: label)
batchInputs.append(featureProvider)}}
let trainingData = MLArrayBatchProvider(array: batchInputs)
When calling the MLUpdateTask as follows, the context.model from completionHandler is null.
Unfortunately there is no other Information available from the compiler.
do{
debugPrint(context)
try context.model.write(to: ModelManager.targetURL)
}
catch{
debugPrint("Error saving the model \(error)")
}
})
updateTask.resume()
I get the following error when I want to access the context.model: Thread 5: EXC_BAD_ACCESS (code=1, address=0x0)
Can some1 more experienced tell me how to fix this?
It seems like I am missing some parameters?
I am currently not splitting the Data when training into train and test data. only preprocessing im doing is scaling the image down to 227x227 pixels.
Thanks!
Does anyone have a ready-made script/shortcut like the one shown in the video?
Hi, just got an Apple M3 Pro to try it out on some Jax operations. I see the development is actively ongoing so maybe this error can help.
This is the environment:
Metal device set to: Apple M3 Pro
systemMemory: 18.00 GB
maxCacheSize: 6.00 GB
jax: 0.4.26
jaxlib: 0.4.23
numpy: 1.26.4
python: 3.11.8 | packaged by conda-forge | (main, Feb 16 2024, 20:49:36) [Clang 16.0.6 ]
jax.devices (1 total, 1 local): [METAL(id=0)]
process_count: 1
platform: uname_result(system='Darwin', node='MKFL96VR9YT', release='23.4.0', version='Darwin Kernel Version 23.4.0: Wed Feb 21 21:44:54 PST 2024; root:xnu-10063.101.15~2/RELEASE_ARM64_T6030', machine='arm64')
This is a minimal example which produces an error, I think due to the fft part:
from jax import numpy as np
array = np.ones((16, 16))
np.fft.fft2(array)
This is the full traceback:
Traceback (most recent call last):
File "/Users/user/Downloads/wow.py", line 5, in <module>
np.fft.fft2(array)
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/numpy/fft.py", line 216, in fft2
return _fft_core_2d('fft2', xla_client.FftType.FFT, a, s=s, axes=axes,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/numpy/fft.py", line 210, in _fft_core_2d
return _fft_core(func_name, fft_type, a, s, axes, norm)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/numpy/fft.py", line 102, in _fft_core
transformed = lax.fft(arr, fft_type, tuple(s))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/traceback_util.py", line 179, in reraise_with_filtered_traceback
return fun(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/pjit.py", line 298, in cache_miss
outs, out_flat, out_tree, args_flat, jaxpr, attrs_tracked = _python_pjit_helper(
^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/pjit.py", line 176, in _python_pjit_helper
out_flat = pjit_p.bind(*args_flat, **params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/core.py", line 2788, in bind
return self.bind_with_trace(top_trace, args, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/core.py", line 425, in bind_with_trace
out = trace.process_primitive(self, map(trace.full_raise, args), params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/core.py", line 913, in process_primitive
return primitive.impl(*tracers, **params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/pjit.py", line 1494, in _pjit_call_impl
return xc._xla.pjit(name, f, call_impl_cache_miss, [], [], donated_argnums, # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/pjit.py", line 1471, in call_impl_cache_miss
out_flat, compiled = _pjit_call_impl_python(
^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/pjit.py", line 1406, in _pjit_call_impl_python
lowering_parameters=mlir.LoweringParameters()).compile()
^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/interpreters/pxla.py", line 2369, in compile
executable = UnloadedMeshExecutable.from_hlo(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/interpreters/pxla.py", line 2908, in from_hlo
xla_executable, compile_options = _cached_compilation(
^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/interpreters/pxla.py", line 2718, in _cached_compilation
xla_executable = compiler.compile_or_get_cached(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/compiler.py", line 266, in compile_or_get_cached
return backend_compile(backend, computation, compile_options,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/profiler.py", line 335, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/jaxmetal/lib/python3.11/site-packages/jax/_src/compiler.py", line 238, in backend_compile
return backend.compile(built_c, compile_options=options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
jaxlib.xla_extension.XlaRuntimeError: UNKNOWN: <unknown>:0: error: 'func.func' op One or more function input/output data types are not supported.
<unknown>:0: note: see current operation:
"func.func"() <{arg_attrs = [{mhlo.layout_mode = "default", mhlo.sharding = "{replicated}"}], function_type = (tensor<16x16xf32>) -> tensor<16x16xcomplex<f32>>, res_attrs = [{jax.result_info = "", mhlo.layout_mode = "default"}], sym_name = "main", sym_visibility = "public"}> ({
^bb0(%arg0: tensor<16x16xf32>):
%0 = "mhlo.convert"(%arg0) : (tensor<16x16xf32>) -> tensor<16x16xcomplex<f32>>
%1 = "mhlo.fft"(%0) {fft_length = dense<16> : tensor<2xi64>, fft_type = #mhlo<fft_type FFT>} : (tensor<16x16xcomplex<f32>>) -> tensor<16x16xcomplex<f32>>
"func.return"(%1) : (tensor<16x16xcomplex<f32>>) -> ()
}) : () -> ()
<unknown>:0: error: failed to legalize operation 'func.func'
<unknown>:0: note: see current operation:
"func.func"() <{arg_attrs = [{mhlo.layout_mode = "default", mhlo.sharding = "{replicated}"}], function_type = (tensor<16x16xf32>) -> tensor<16x16xcomplex<f32>>, res_attrs = [{jax.result_info = "", mhlo.layout_mode = "default"}], sym_name = "main", sym_visibility = "public"}> ({
^bb0(%arg0: tensor<16x16xf32>):
%0 = "mhlo.convert"(%arg0) : (tensor<16x16xf32>) -> tensor<16x16xcomplex<f32>>
%1 = "mhlo.fft"(%0) {fft_length = dense<16> : tensor<2xi64>, fft_type = #mhlo<fft_type FFT>} : (tensor<16x16xcomplex<f32>>) -> tensor<16x16xcomplex<f32>>
"func.return"(%1) : (tensor<16x16xcomplex<f32>>) -> ()
}) : () -> ()
I'd be happy running more tests should you need them, I'm new to this, so not sure which just yet.
Many thanks!!
Hey, i just created and trained an MLImageClassifier via the MLImageclassifier.train() method (https://developer.apple.com/documentation/createml/mlimageclassifier/train(trainingdata:parameters:sessionparameters:))
For my Trainingdata (MLImageclassifier.DataSource) i am using my directoy structure, so i got an images folder with subfolders of person1, person2, person3 etc. which contain images of the labeled persons (https://developer.apple.com/documentation/createml/mlimageclassifier/datasource/labeleddirectories(at:))
I am saving the checkpoints and sessions in my appdirectory, so i can create an MLIMageClassifier from an exisiting MLSession and/or MLCheckpoint.
My question is: is there any way to add new labels, optimally from my directoy strucutre, to an MLImageClassifier which i create from an existing MLCheckpoint/MLSession?
So like adding a person4 and training my pretrained Classifier with only that person4.
Or is it simply not possible and i have to train from the beginning everytime i want to add a new label?
Unfortunately i cannot find anything in the API.
Thanks!
Hey,
im training an MLImageClassifier via the train()-method:
guard let job = try? MLImageClassifier.train(trainingData: trainingData, parameters: modelParameter, sessionParameters: sessionParameters) else{
debugPrint("Training failed")
return
}
Unfortunately the metrics of my MLProgress, which is created from the returning MLJob while training are empty.
Code for listening on Progress:
job.progress.publisher(for: \.fractionCompleted)
.sink{[weak job] fractionCompleted in
guard let job = job else {
debugPrint("failure in creating job")
return
}
guard let progress = MLProgress(progress: job.progress) else {
debugPrint("failure in creating progress")
return
}
print("ProgressPROGRESS: \(progress)")
print("Progress: \(fractionCompleted)")
}
.store(in: &subscriptions)
Printing the Progress ends in:
MLProgress(elapsedTime: 2.2328420877456665, phase: CreateML.MLPhase.extractingFeatures, itemCount: 32, totalItemCount: Optional(39), metrics: [:])
Got the Same result when listening to MLCheckpoints, Metrics are empty aswell:
MLCheckpoint(url: URLPATH.checkpoint, phase: CreateML.MLPhase.extractingFeatures, iteration: 32, date: 2024-04-18 11:21:18 +0000, metrics: [:])
Can some1 tell me how I can access the metrics while training?
Thanks!
Hello Developers,
We are trying to convert Pytorch models to CoreML using coremltools,
while converting we used jit.trace to create trace of model where we encountered a warning that if model has controlflow and conditions it is not advisable to use trace instead convert into TorchScript using jit.script,
However after successful conversion of model into TorchScript, Now in the next step of conversion from TorchScript to CoreML here is the error we are getting when we tried to convert to coremltools python package.
This root error is so abstract that we are not able to trace-back from where its occurring.
AssertionError: Item selection is supported only on python list/tuple objects
We trying to add this above error prompt into ChatGPT and we get something like the below response from ChatGPT. But unfortunately it's not working.
The error indicates that the Core ML converter encountered a TorchScript operation involving item selection (indexing or slicing) on an object that it doesn't recognize as a Python list or tuple. The converter supports item selection only on these Python container types. This could happen if your model uses indexing on tensors or other types not recognized as list or tuple by the Core ML tools. You may need to revise the TorchScript code to ensure it only performs item selection on supported types or adjust the way tensors are indexed.
I have a trained model to identify squats (good & bad repetitions). It seems to be working perfectly in CreateML when I preview it with some test data, although once I add it to my app the model seems to be inaccurate and the majority of the time mixes up the actions. Does anyone know if the issue is code related or is it something to do with the model itself and how it analyses live data?
Below I have added one of my functions for "Good Squats" which most of the time doesn't even get called (even with lower confidence). The majority of the time the model classes everything as a bad squat even though it is clearly not.
Could the problem be that my dataset doesn't have enough videos?
print("GoodForm")
squatDetected = true
DispatchQueue.main.asyncAfter(deadline: .now() + 1.5) {
self.squatDetected = false
}
DispatchQueue.main.async {
self.showGoodFormAlert(with: confidence)
AudioServicesPlayAlertSound(SystemSoundID(1322))
}
}
Any help would be appreciated.