I'm excited to see that SAM has been successfully integrated with Prodigy—excellent work!
I am currently exploring the possibility of integrating MedSAM, the medical variant of SAM, for enhanced labelling capabilities. MedSAM, detailed here: GitHub - bowang-lab/MedSAM: Segment Anything in Medical Images, offers features specifically tailored to medical applications. I'm particularly interested in its potential for object segmentation in endoscopic videos when combined with Prodigy.
Could you provide any guidance or suggestions on how to achieve this integration? Any assistance would be highly valued.
Thanks for the nice words
MedSam (at least medsam_vit_b.pth checkpoint) should pretty much work out of the box?
I have successfully run it by specifying the model's registry id as vit_b and providing the path to the pre-downloaded checkpoint like so:
Thanks @magdaaniol for the guidance. So excited to see MedSAM can be available so easily.
When I run the command, I am getting the following error:
python -m prodigy segment.image.manual test_segment ./frames medsam_vit_b.pth --model-type medsam_vit_b --label A,B -R
2024-05-19 00:10:27.176424: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-05-19 00:10:27.212225: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-19 00:10:27.771976: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-05-19 00:10:28.593596: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1960] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
✘ Can't find recipe or command 'segment.image.manual'.
Run prodigy --help to see available options. If you're using a custom recipe,
provide the path to the Python file using the -F argument.
Oh, it looks like it's impossible to uninstall the older version of one of the dependencies because it's been installed with conda (not pip). Could you try installing Prodigy and the Prodigy-Segment plugin in a fresh virtual environment.
That is:
In your project folder (create a fresh one too if possible):
Alternatively, you could also try to uninstall opencv-python using conda so conda uninstall opencv-python but not sure if it's perhaps needed somwhere else in your setup. This is why I would strongly recommend starting with a fresh virtual environment for a project with Prodigy and Prodigy-Segment. That would be the cleanest way for sure.
Dear @magdaaniol
I have tried as you proposed to install it in virtual environment. It installed smoothly however, you I am getting the following error:
python -m prodigy segment.image.manual test_segment ./frames medsam_vit_b.pth --model-type medsam_vit_b --label A,B -R
Traceback (most recent call last):
File "/home/big-deal/mambaforge/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/big-deal/mambaforge/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/big-deal/mlworks/.venv/lib/python3.10/site-packages/prodigy/__main__.py", line 53, in <module>
registry.recipes.get_entry_points()
File "/home/big-deal/mlworks/.venv/lib/python3.10/site-packages/catalogue/__init__.py", line 125, in get_entry_points
result[entry_point.name] = entry_point.load()
File "/home/big-deal/mambaforge/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
module = import_module(match.group('module'))
File "/home/big-deal/mambaforge/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/home/big-deal/mlworks/.venv/lib/python3.10/site-packages/prodigy_segment/__init__.py", line 12, in <module>
from prodigy.core import Arg, recipe, Controller
ImportError: cannot import name 'Arg' from 'prodigy.core' (/home/big-deal/mlworks/.venv/lib/python3.10/site-packages/prodigy/core.cpython-310-x86_64-linux-gnu.so)
Can you please guide me on how to resolve this?
Thanks
hmm it looks like Prodigy and the plugin are both in the same .venv now (/home/big-deal/mlworks/.venv/
With this virtual environment activated could you run:
Right, I'm afraid you'll need Prodigy >= 1.14.0 to be able to use the plugin. In 1.14.0 we made significant update to the internals which includes moving to radicli for CLI management (hence the import error). This alone would be fairly easy to "undo" and make the plugin code backward compatible but there are also changes to the types which are more involved.
I'll pin Prodigy version on the plugin to avoid this in future.
It might be that your license has expired? If you purchased more than a year ago than it won't let you download the versions that are newer than the ones released within a year from your purchase
You can check which Prodigy versions you're license is entitled to by accessing the download URL with your browser: https://XXXX-XXXX-XXXX-XXXX@download.prodi.gy.
Just a comment on your command, you probably should put the version in quotes so:
