As the title suggests I am using AVAudioEngine for SpeechRecognition input & AVAudioPlayer for sound output. Apple says in this talk https://developer.apple.com/videos/play/wwdc2019/510 that the setVoiceProcessingEnabled function very usefully cancels the output from speaker to the mic. I set voiceProcessing on the Input and output nodes. It seems to work however the volume is low, even when the system volume is turned up. Any solution to this would be much appreciated.

What is the accuracy and resolution of the angles measured using Vision?

I am trying to parse text from an image, split it into words and store the words in a String array. Additionally I want to store the bounding box of each recognized word. My code works but for some reason the bounding boxes of words that are not separated by a space but by an apostrophe come out wrong. Here is the complete code of my VNRecognizeTextRequestHander: let request = VNRecognizeTextRequest { request, error in guard let observations = request.results as? [VNRecognizedTextObservation] else { return } // split recognized text into words and store each word with corresponding observation let wordObservations = observations.flatMap { observation in observation.topCandidates(1).first?.string.unicodeScalars .split(whereSeparator: { CharacterSet.letters.inverted.contains($0) }) .map { (observation, $0) } ?? [] } // store recognized words as strings recognizedWords = wordObservations.map { (observation, word) in String(word) } // calculate bounding box for each word recognizedWordRects = wordObservations.map { (observation, word) in guard let candidate = observation.topCandidates(1).first else { return .zero } let stringRange = word.startIndex..<word.endIndex guard let rect = try? candidate.boundingBox(for: stringRange)?.boundingBox else { return .zero } let bottomLeftOriginRect = VNImageRectForNormalizedRect(rect, Int(captureRect.width), Int(captureRect.height)) // adjust coordinate system to start in top left corner let topLeftOriginRect = CGRect(origin: CGPoint(x: bottomLeftOriginRect.minX, y: captureRect.height - bottomLeftOriginRect.height - bottomLeftOriginRect.minY), size: bottomLeftOriginRect.size) print("BoundingBox for word '\(String(word))': \(topLeftOriginRect)") return topLeftOriginRect } } And here's an example for what's happening. When I'm processing the following image: the code above produces the following output: BoundingBox for word 'In': (23.00069557577264, 5.718113962610181, 45.89460636656961, 32.78087073878238) BoundingBox for word 'un': (71.19064286904202, 6.289275587192936, 189.16024359557852, 34.392966621800475) BoundingBox for word 'intervista': (71.19064286904202, 6.289275587192936, 189.16024359557852, 34.392966621800475) BoundingBox for word 'del': (262.64622870703477, 8.558512219726875, 54.733978711037985, 32.79967358237818) Notice how the bounding boxes of the words 'un' and 'intervista' are exactly the same. This happens consistently for words that are separated by an apostrophe. Why is that? Thank you for any help Elias

Running the sample Python keras-ocr example on M3 Max returns incorrect results if tensorflow-metal is installed. Code Example: https://keras-ocr.readthedocs.io/en/latest/examples/using_pretrained_models.html Note: https://upload.wikimedia.org/wikipedia/commons/e/e8/FseeG2QeLXo.jpg not found. Line commented out. Without tensorflow-metal (Correct results): ['toodstande', 's', 'somme', 'srny', 'squadron', 'ds', 'quentn', 'snhnen', 'bnpnone', 'sasne', 'taing', 'yeoms', 'sry', 'the', 'royal', 'wessex', 'yeomanry', 'regiment', 'yeomanry', 'wests', 'south', 'the', 'now', 'recruiting', 'arm', 'blon', 'wxybsqipsacomodn', 'email', '438300', '01722'] ['banana', 'union', 'no', 'no', 'software', 'patents'] With tensorflow-metal (Incorrect results): ['sddoooo', '', 'eamnooss', 'xynrr', 'daanues', 'idd', 'innee', 'iiiinus', 'tnounppanab', 'inla', 'ppnt', 'mmnooexyy', 'yyr', 'ehhtt', 'laayvyoorr', 'xeseww', 'rinamoevy', 'tnemiger', 'yrnamoey', 'sstseww', 'htuwlos', 'fefeahit', 'wwoniia', 'turceedrr', 'ymmrira', 'atate', 'prasbyxwr', 'liamme', '00338803144', '22277100'] ['annnaab', 'noolinnu', 'oon', 'oon', 'wttffoos', 'sttneettaap'] Logs: With tensorflow-metal (Incorrect results) (.venv) <REDACTED> % pip3 install -U tensorflow-metal Collecting tensorflow-metal Using cached tensorflow_metal-1.1.0-cp311-cp311-macosx_12_0_arm64.whl.metadata (1.2 kB) Requirement already satisfied: wheel~=0.35 in ./.venv/lib/python3.11/site-packages (from tensorflow-metal) (0.42.0) Requirement already satisfied: six>=1.15.0 in ./.venv/lib/python3.11/site-packages (from tensorflow-metal) (1.16.0) Using cached tensorflow_metal-1.1.0-cp311-cp311-macosx_12_0_arm64.whl (1.4 MB) Installing collected packages: tensorflow-metal Successfully installed tensorflow-metal-1.1.0 (.venv) <REDACTED> % python3 keras-ocr-bug.py Looking for <REDACTED>/.keras-ocr/craft_mlt_25k.h5 2023-12-16 22:05:05.452493: I metal_plugin/src/device/metal_device.cc:1154] Metal device set to: Apple M3 Max 2023-12-16 22:05:05.452532: I metal_plugin/src/device/metal_device.cc:296] systemMemory: 64.00 GB 2023-12-16 22:05:05.452545: I metal_plugin/src/device/metal_device.cc:313] maxCacheSize: 24.00 GB 2023-12-16 22:05:05.452591: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:306] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support. 2023-12-16 22:05:05.452609: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:272] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>) WARNING:tensorflow:From <REDACTED>/.venv/lib/python3.11/site-packages/tensorflow/python/util/dispatch.py:1260: resize_bilinear (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.image.resize(...method=ResizeMethod.BILINEAR...)` instead. Looking for <REDACTED>/.keras-ocr/crnn_kurapan.h5 2023-12-16 22:05:07.526354: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:117] Plugin optimizer for device_type GPU is enabled. 1/1 [==============================] - 1s 855ms/step 2/2 [==============================] - 1s 140ms/step ['sddoooo', '', 'eamnooss', 'xynrr', 'daanues', 'idd', 'innee', 'iiiinus', 'tnounppanab', 'inla', 'ppnt', 'mmnooexyy', 'yyr', 'ehhtt', 'laayvyoorr', 'xeseww', 'rinamoevy', 'tnemiger', 'yrnamoey', 'sstseww', 'htuwlos', 'fefeahit', 'wwoniia', 'turceedrr', 'ymmrira', 'atate', 'prasbyxwr', 'liamme', '00338803144', '22277100'] ['annnaab', 'noolinnu', 'oon', 'oon', 'wttffoos', 'sttneettaap'] Logs: Valid results, without tensorflow-metal (.venv) <REDACTED> % pip3 uninstall tensorflow-metal Found existing installation: tensorflow-metal 1.1.0 Uninstalling tensorflow-metal-1.1.0: Would remove: <REDACTED>/.venv/lib/python3.11/site-packages/tensorflow-plugins/* <REDACTED>/.venv/lib/python3.11/site-packages/tensorflow_metal-1.1.0.dist-info/* Proceed (Y/n)? Y Successfully uninstalled tensorflow-metal-1.1.0 (.venv) <REDACTED> % python3 keras-ocr-bug.py Looking for <REDACTED>/.keras-ocr/craft_mlt_25k.h5 WARNING:tensorflow:From <REDACTED>/.venv/lib/python3.11/site-packages/tensorflow/python/util/dispatch.py:1260: resize_bilinear (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.image.resize(...method=ResizeMethod.BILINEAR...)` instead. Looking for <REDACTED>/.keras-ocr/crnn_kurapan.h5 1/1 [==============================] - 7s 7s/step 2/2 [==============================] - 1s 71ms/step ['toodstande', 's', 'somme', 'srny', 'squadron', 'ds', 'quentn', 'snhnen', 'bnpnone', 'sasne', 'taing', 'yeoms', 'sry', 'the', 'royal', 'wessex', 'yeomanry', 'regiment', 'yeomanry', 'wests', 'south', 'the', 'now', 'recruiting', 'arm', 'blon', 'wxybsqipsacomodn', 'email', '438300', '01722'] ['banana', 'union', 'no', 'no', 'software', 'patents']

Hi, I think many of us would love to be able to use our GPUs for Jax on the new Apple Silicon devices, but currently, the Jax-metal plugin is, for all effects and purposes, broken. Is it still under active development? Is there a planned release for a new version? thanks!

I seem to be having some trouble running the example app from the WWDC 2023 session on 3D Body Pose Detection (this one). I'm getting an issue about the request revision being incompatible, I tried searching the API documentation for any configuration that has been changed or introduced but to no avail. I also couldn't find much online for it. Is this a known issue? Or am I doing something wrong? Error in question: Unable to perform the request: Error Domain=com.apple.Vision Code=16 "VNDetectHumanBodyPose3DRequest does not support VNDetectHumanBodyPose3DRequestRevision1" UserInfo={NSLocalizedDescription=VNDetectHumanBodyPose3DRequest does not support VNDetectHumanBodyPose3DRequestRevision1}. Code Snippet: guard let assetURL = fileURL else { return } let request = VNDetectHumanBodyPose3DRequest() self.fileURL = assetURL let requestHandler = VNImageRequestHandler(url: assetURL) do { try requestHandler.perform([request]) if let returnedObservation = request.results?.first { Task { @MainActor in self.humanObservation = returnedObservation } } } catch { print("Unable to perform the request: \(error).") } Thank you for any and all advice!

Hi, I have been trying to connect my microphone on my reason studio for days now without any outcome. So I was asked to download ASIO Driver on my mac. I realised that I have an IAC driver. I need help on downloading the Asio driver and wish to know if there will be a problem running it with the IAC driver. I also just upgraded without knowing from ventura to sonoma. Am using an audio iinterface(Focusrite scarlett solo to connect to the reason application and my mic is nt1-a rode but I can get a sound but cannot record. Will be overwhelmed if I can get help from here. Thanks

Hello, I followed the instructions provided here: https://developer.apple.com/metal/tensorflow-plugin/ and while trying to run the example I am getting following error: otFoundError: dlopen(/Users/nedimhadzic/venv-metal/lib/python3.11/site-packages/tensorflow-plugins/libmetal_plugin.dylib, 0x0006): Symbol not found: __ZN10tensorflow16TensorShapeProtoC1ERKS0_ Referenced from: <C62E0AB4-567E-3E14-8F96-9F07A746C4DC> /Users/nedimhadzic/venv-metal/lib/python3.11/site-packages/tensorflow-plugins/libmetal_plugin.dylib Expected in: <FFF31651-3926-3E79-A442-143B7156FB13> /Users/nedimhadzic/venv-metal/lib/python3.11/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so tensorflow: 2.15.0 tensorlow-metal: 1.0.0 macos: 14.2.1 Intel CPU and AMD Radeon Pro 5500M Any idea? Regards, Nedim

Currently in South Korea, due to my personal experiences with what seems like warranty but isn't, and the operation of a ruthless Genius Bar, I feel compelled to purchase the officially released Vision Pro. I'd like to discuss with other developers here about their thoughts on the release schedule. The product launched in the USA in February, but I'm curious about the months following for the secondary and tertiary launch countries. Naturally, we'll know the launch is imminent when local staff are summoned to the headquarters for training. However, the urgency for localized services, development, and personal purchase is growing on my mind.

Hi Apple Team, We have a technical query regarding one feature- Audio Recognition and Live captioning. We are developing an app for deaf community to avoid communication barriers. We want to know if there is any possibility to recognize the sound from other applications in an iPhone and show live captions in our application (based on iOS).

Here is my environment : python==3.9.0 tensorflow==2.9.0 os==Sonoma 14.2 (23C64) Error : Translated Report (Full Report Below) Process: Python [10330] Path: /Library/Frameworks/Python.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python Identifier: org.python.python Version: 3.9.0 (3.9.0) Code Type: X86-64 (Translated) Parent Process: Python [8039] Responsible: Terminal [779] User ID: 501 Date/Time: 2023-12-30 22:31:38.4916 +0530 OS Version: macOS 14.2 (23C64) Report Version: 12 Anonymous UUID: F7E462E7-6380-C3DA-E2EC-5CF01A61D195 Sleep/Wake UUID: 50F32A2D-8CFA-4117-8048-D9CF76E24F26 Time Awake Since Boot: 29000 seconds Time Since Wake: 2193 seconds System Integrity Protection: enabled Notes: PC register does not match crashing frame (0x0 vs 0x10CEDE6D9) Crashed Thread: 0 Dispatch queue: com.apple.main-thread Exception Type: EXC_BAD_INSTRUCTION (SIGILL) Exception Codes: 0x0000000000000001, 0x0000000000000000 Termination Reason: Namespace SIGNAL, Code 4 Illegal instruction: 4 Terminating Process: exc handler [10330] Error Formulating Crash Report: PC register does not match crashing frame (0x0 vs 0x10CEDE6D9) Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 _cpu_feature_guard.so 0x10cede6d9 _GLOBAL__sub_I_cpu_feature_guard.cc + 9 1 dyld 0x2026a3fca invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const::$_0::operator()() const + 182 2 dyld 0x2026e5584 invocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 133 3 dyld 0x2026d9913 invocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 543 4 dyld 0x20268707f dyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const + 249 5 dyld 0x2026d8adc dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 176 6 dyld 0x2026db104 dyld3::MachOFile::forEachInitializerPointerSection(Diagnostics&, void (unsigned int, unsigned int, bool&) block_pointer) const + 116 7 dyld 0x2026e52ba dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 390 8 dyld 0x2026a0cfc dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const + 222 9 dyld 0x2026a65cb dyld4::JustInTimeLoader::runInitializers(dyld4::RuntimeState&) const + 21 10 dyld 0x2026a0ef1 dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 181 11 dyld 0x2026a4040 dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const::$_1::operator()() const + 98 12 dyld 0x2026a0f87 dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const + 93 13 dyld 0x2026bdc65 dyld4::APIs::dlopen_from(char const*, int, void*) + 935 14 _ctypes.cpython-39-darwin.so 0x10ac20962 py_dl_open + 162 15 Python 0x10b444f2d cfunction_call + 125 16 Python 0x10b40625d _PyObject_MakeTpCall + 365 17 Python 0x10b4dc8fc call_function + 876 18 Python 0x10b4d9e2b _PyEval_EvalFrameDefault + 25371 19 Python 0x10b4dd563 _PyEval_EvalCode + 2611 20 Python 0x10b4069b1 _PyFunction_Vectorcall + 289 21 Python 0x10b4060b5 _PyObject_FastCallDictTstate + 293 22 Python 0x10b406c98 _PyObject_Call_Prepend + 152 23 Python 0x10b4601e5 slot_tp_init + 165 24 Python 0x10b45b699 type_call + 345 ... Thread 1:: com.apple.rosetta.exceptionserver 0 runtime 0x7ff7fffaf294 0x7ff7fffab000 + 17044 Thread 2:: /Reaper 0 ??? 0x7ff8aa35ea78 ??? 1 libsystem_kernel.dylib 0x7ff819da46fa kevent + 10 2 libzmq.5.dylib 0x10bf038f6 zmq::kqueue_t::loop() + 278 3 libzmq.5.dylib 0x10bf31a59 zmq::worker_poller_base_t::worker_routine(void) + 25 4 libzmq.5.dylib 0x10bf7854c thread_routine(void*) + 300 5 libsystem_pthread.dylib 0x7ff819ddf202 _pthread_start + 99 6 libsystem_pthread.dylib 0x7ff819ddabab thread_start + 15 Thread 3:: /0 0 ??? 0x7ff8aa35ea78 ??? 1 libsystem_kernel.dylib 0x7ff819da46fa kevent + 10 2 libzmq.5.dylib 0x10bf038f6 zmq::kqueue_t::loop() + 278 3 libzmq.5.dylib 0x10bf31a59 zmq::worker_poller_base_t::worker_routine(void) + 25 4 libzmq.5.dylib 0x10bf7854c thread_routine(void*) + 300 5 libsystem_pthread.dylib 0x7ff819ddf202 _pthread_start + 99 6 libsystem_pthread.dylib 0x7ff819ddabab thread_start + 15 Thread 4: 0 ??? 0x7ff8aa35ea78 ??? 1 libsystem_kernel.dylib 0x7ff819da46fa kevent + 10 2 select.cpython-39-darwin.so 0x10ab95dc3 select_kqueue_control + 915 3 Python 0x10b40f11f method_vectorcall_FASTCALL + 335 4 Python 0x10b4dc86c call_function + 732 5 Python 0x10b4d9d72 _PyEval_EvalFrameDefault + 25186 6 Python 0x10b4dd563 _PyEval_EvalCode + 2611 7 Python 0x10b4069b1 _PyFunction_Vectorcall + 289 ...

Application is getting Crashed: AXSpeech EXC_BAD_ACCESS KERN_INVALID_ADDRESS 0x000056f023efbeb0 Crashed: AXSpeech 0 libobjc.A.dylib 0x4820 objc_msgSend + 32 1 libsystem_trace.dylib 0x6c34 _os_log_fmt_flatten_object + 116 2 libsystem_trace.dylib 0x5344 _os_log_impl_flatten_and_send + 1884 3 libsystem_trace.dylib 0x4bd0 _os_log + 152 4 libsystem_trace.dylib 0x9c48 _os_log_error_impl + 24 5 TextToSpeech 0xd0a8c _pcre2_xclass_8 6 TextToSpeech 0x3bc04 TTSSpeechUnitTestingMode 7 TextToSpeech 0x3f128 TTSSpeechUnitTestingMode 8 AXCoreUtilities 0xad38 -[NSArray(AXExtras) ax_flatMappedArrayUsingBlock:] + 204 9 TextToSpeech 0x3eb18 TTSSpeechUnitTestingMode 10 TextToSpeech 0x3c948 TTSSpeechUnitTestingMode 11 TextToSpeech 0x48824 AXAVSpeechSynthesisVoiceFromTTSSpeechVoice 12 TextToSpeech 0x49804 AXAVSpeechSynthesisVoiceFromTTSSpeechVoice 13 Foundation 0xf6064 __NSThreadPerformPerform + 264 14 CoreFoundation 0x37acc CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION + 28 15 CoreFoundation 0x36d48 __CFRunLoopDoSource0 + 176 16 CoreFoundation 0x354fc __CFRunLoopDoSources0 + 244 17 CoreFoundation 0x34238 __CFRunLoopRun + 828 18 CoreFoundation 0x33e18 CFRunLoopRunSpecific + 608 19 Foundation 0x2d4cc -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 212 20 TextToSpeech 0x24b88 TTSCFAttributedStringCreateStringByBracketingAttributeWithString 21 Foundation 0xb3154 NSThread__start + 732 com.livingMedia.AajTakiPhone_issue_3ceba855a8ad2d1af83655803dc13f70_crash_session_9081fa41ced440ae9a57c22cb432f312_DNE_0_v2_stacktrace.txt 22 libsystem_pthread.dylib 0x24d4 _pthread_start + 136 23 libsystem_pthread.dylib 0x1a10 thread_start + 8

Hi, In Xcode 14 I was able to train linear regression models with Create ML using large CSV files (I tested on about 30000 items and 5 features): However, in Xcode 15 (I tested on 15.0.1 and 15.1), the training continuously stays in the "Processing" state: When using a dataset with 900 items, everything works fine. I filed a feedback for this issue: FB13516799. Does anybody else have this issue / can reproduce it?

Hello, I got a brand new MacBook M3 Pro and trying to configure Tensorflow w/ GPU support. I followed instructions provided at https://developer.apple.com/metal/tensorflow-plugin/ step by step. Unfortunately, even after creating/recreating/installing/uninstalling TensorFlow the problem is not getting resolved as Python crashes. I cannot get past that point to try Jupyter notebook. Here is the error ask the versions in "tf" environment. I already spent entire Saturday yesterday and so far no progress. Can someone tell me what is going on? Python 3.11.7 (main, Dec 4 2023, 18:10:11) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin Type "help", "copyright", "credits" or "license" for more information. import tensorflow as tf 2024-01-07 11:44:04.893581: F tensorflow/c/experimental/stream_executor/stream_executor.cc:743] Non-OK-status: stream_executor::MultiPlatformManager::RegisterPlatform( std::move(cplatform)) status: INTERNAL: platform is already registered with name: "METAL" [1] 1797 abort /opt/homebrew/bin/python3 ❯ python -m pip list | grep tensorflow tensorflow 2.15.0 tensorflow-estimator 2.15.0 tensorflow-io-gcs-filesystem 0.34.0 tensorflow-macos 2.15.0 tensorflow-metal 1.1.0 ❯ python --version Python 3.11.7 OS is Sonoma 14.2.1 Thanks Sohail

Hello I use Mac Pro M2 16GB This is my code. It is very basic code. `model = Sequential() model.add(LSTM(units=50, input_shape=(X_train.shape[1], X_train.shape[2]))) model.add(Dense(units=1)) model.compile(optimizer='adam', loss='mse') model.fit(X_train, y_train, epochs=50, batch_size=16) train_predict = model.predict(X_train) test_predict = model.predict(X_test) train_predict = scaler.inverse_transform(train_predict) y_train = scaler.inverse_transform(y_train) test_predict = scaler.inverse_transform(test_predict) y_test = scaler.inverse_transform(y_test) When I try to execute this code, anaconda gives the following error I metal_plugin/src/device/metal_device.cc:1154] Metal device set to: Apple M2 I metal_plugin/src/device/metal_device.cc:296] systemMemory: 16.00 GB I metal_plugin/src/device/metal_device.cc:313] maxCacheSize: 5.33 GB I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:306] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support. : I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:272] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: ) I can't find any solution, could you help me Thank you

I am working on a design that requires connecting an ios device to two audio output devices specifically headphones and a speaker. I want the audio driver to switch output device without user action. Is this manageable via ios SDK?

Hi, there seems to be a difference in behavior when running inference on a trained Keras model using the model __call__ method vs. using the predict or predict_on_batch methods. This only happens when using the GPU for inference and it seems that for certain sequence of operations and float types the 'relu' activation doesn't work as expected and seems to do nothing. I can replicate the problem with the following code (it would only fail with 'relu' activation and tf.float16 and tf.float32 types, while it works fine with tf.float64). import tensorflow as tf import numpy as np DATA_LENGTH = 16 DENSE_WIDTH = 16 BATCH_SIZE = 8 DTYPE = tf.float32 ACTIVATION = 'relu' def TestModel(): inputs = tf.keras.Input(DATA_LENGTH, dtype=DTYPE) u = tf.keras.layers.Dense(DENSE_WIDTH, activation=ACTIVATION, dtype=DTYPE)(inputs) # u = tf.maximum(u, 0.0) output = u*tf.constant(1.0, dtype=DTYPE) model = tf.keras.Model(inputs, output, name="TestModel") return model model = TestModel() model.compile() x = np.random.uniform(size=(BATCH_SIZE, DATA_LENGTH)).astype(DTYPE.as_numpy_dtype) with tf.device('/GPU:0'): out_gpu_call = model(x, training=False) out_gpu_predict = model.predict_on_batch(x) with tf.device('/CPU:0'): out_cpu_call = model(x, training=False) out_cpu_predict= model.predict_on_batch(x) print(f'\nDTYPE {DTYPE}, ACTIVATION: {ACTIVATION}') print("\tMean Abs. Difference GPU (__call__ vs. predict):", np.mean(np.abs(out_gpu_call - out_gpu_predict))) print("\tMean Abs. Difference CPU (__call__ vs. predict):", np.mean(np.abs(out_cpu_call - out_cpu_predict))) print("\tMean Abs. Difference GPU-CPU __call__:", np.mean(np.abs(out_gpu_call - out_cpu_call))) print("\tMean Abs. Difference GPU-CPU predict():", np.mean(np.abs(out_gpu_predict - out_cpu_predict))) The code above produces for example the following output: DTYPE <dtype: 'float32'>, ACTIVATION: relu Mean Abs. Difference GPU (__call__ vs. predict): 0.1955472 Mean Abs. Difference CPU (__call__ vs. predict): 0.0 Mean Abs. Difference GPU-CPU __call__: 1.3573299e-08 Mean Abs. Difference GPU-CPU predict(): 0.1955472 And the results for the GPU are: out_gpu_call <tf.Tensor: shape=(8, 16), dtype=float32, numpy= array([[0.1496982 , 0. , 0. , 0.73772687, 0.26131183, 0.27757105, 0. , 0. , 0. , 0. , 0. , 0.4164225 , 1.0367445 , 0. , 0.5860609 , 0. ], ... out_gpu_predict array([[ 1.49698198e-01, -3.48425686e-01, -2.44667321e-01, 7.37726867e-01, 2.61311829e-01, 2.77571052e-01, -2.26729304e-01, -1.06500387e-01, -3.66294265e-01, -2.93850392e-01, -4.51043218e-01, 4.16422486e-01, 1.03674448e+00, -1.39347658e-01, 5.86060882e-01, -2.05334812e-01], ... Upon inspection of the results it seems that the problem is that the 'relu' activation is not setting the values < 0 to 0 when calling predict_on_batch. When uncommenting the # u = tf.maximum(u, 0.0) line after the Dense layer there is no difference between the two calls (as should be expected). It also happens that removing the multiplication by a constant after the Dense layer, output = u*tf.constant(1.0, dtype=DTYPE) makes the problem dissappear (even when leaving the # u = tf.maximum(u, 0.0) line commented). This is running with the following setup: MacBook Pro, Apple M2 Max chip, macOS Sonoma 14.2 tf version 2.15.0 tensorflow-metal 1.1.0 Python 3.10.13

Tensorflow-Metal training got an increasing loss in CNN. But same codes run correctly after pip uninstall tensorflow-metal

I'm going to the U.S. to buy a vision pro, does anyone have any information about where they sell it? Will it be sold in Hawaii by any chance? For now, I'm thinking about New York.

I'm currently building an iOS app that requires the ability to detect a person's height with a live video stream. The new VNDetectHumanBodyPose3DRequest is exactly what I need but the observations I'm getting back are very inconsistent and unreliable. When I say inconsistent, I mean the values never seem to settle and they can fluctuate anywhere from 5 '4" to 10'1" (I'm about 6'0"). In terms of unreliable, I have once seen a value that closely matches my height but I rarely see any values that are close enough (within an inch) of the ground truth. In terms of my code, I'm not doing any fancy. I'm first opening a LiDAR stream on my iPhone Pro 14: guard let videoDevice = AVCaptureDevice.default(.builtInLiDARDepthCamera, for: .video, position: .back) else { return } guard let videoDeviceInput = try? AVCaptureDeviceInput(device: videoDevice) else { return } guard captureSession.canAddInput(videoDeviceInput) else { return } captureSession.addInput(videoDeviceInput) I'm then creating an output synchronizer so I can get both image and depth data at the same time: videoDataOutput = AVCaptureVideoDataOutput() captureSession.addOutput(videoDataOutput) depthDataOutput = AVCaptureDepthDataOutput() depthDataOutput.isFilteringEnabled = true captureSession.addOutput(depthDataOutput) outputVideoSync = AVCaptureDataOutputSynchronizer(dataOutputs: [depthDataOutput, videoDataOutput]) Finally, my delegate function that handles the synchronizer is roughly: fileprivate func perform3DPoseRequest(cmSampleBuffer: CMSampleBuffer, depthData: AVDepthData) { let imageRequestHandler = VNImageRequestHandler(cmSampleBuffer: cmSampleBuffer, depthData: depthData, orientation: .up) let request = VNDetectHumanBodyPose3DRequest() do { // Perform the body pose request. try imageRequestHandler.perform([request]) if let observation = request.results?.first { if (observation.heightEstimation == .measured) { print("Body height (ft) \(formatter.string(fromMeters: Double(observation.bodyHeight))) (m): \(observation.bodyHeight)") ... I'd appreciate any help determining how to get accurate results from the observation's bodyHeight. Thanks!