Trying to get THIS repo working on macbook
Cloned repo – tried to install environment.yaml and got
ResolvePackageNotFound:
- cudatoolkit=11.3
Some of the reading I’ve done about getting diffusion working on apple silicon has said these devices can’t use cuda anyway so I commented out the line for this package in the yaml file.
There were other problems in installing this on the PIP side of things, I couldnt isolate the specific package causing a problem when installing from the yaml file so i installed each manually using pip in the terminal.
I think there may have been an issue with grpcio – which is installed when i try to install pytorch-lightning==1.4.2, i tried upgrading setuptools with pip which doesnt seem to have fixed it. I’m going to try following THIS solution and see if that helps.
Tried running this from the issue mentioned above, the bottom line is from the next bookmark mentioned below
export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1
pip install pytorch_lightning
export PYTORCH_ENABLE_MPS_FALLBACK=1
In addition to the changes above, the following changes were made to ldm/models/diffusion/ddim.py
@@ -17,9 +17,10 @@ class DDIMSampler(object):
self.schedule = schedule
def register_buffer(self, name, attr):
+ device = torch.device("cuda") if torch.cuda.is_available() else torch.device("mps")
if type(attr) == torch.Tensor:
- if attr.device != torch.device("cuda"):
- attr = attr.to(torch.device("cuda"))
+ if attr.device != device:
+ attr = attr.type(torch.float32).to(device).contiguous() #attr.to(device)
setattr(self, name, attr)
and I also made a copy of scripts/img2img.py renamed to scripts/changedimg2img.py and made the following changes
@@ -40,7 +40,7 @@ def load_model_from_config(config, ckpt, verbose=False):
print("unexpected keys:")
print(u)
- model.cuda()
+ model.to(device='mps')
model.eval()
return model
@@ -199,11 +199,11 @@ def main():
config = OmegaConf.load(f"{opt.config}")
model = load_model_from_config(config, f"{opt.ckpt}")
- device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
+ device = torch.device("cuda") if torch.cuda.is_available() else torch.device("mps")
model = model.to(device)
Apple have also converted some ckpts to work better, one was available
After making these changes and running something like python scripts/changedimg2img.py --prompt "Corgi dressed as a goth" --init-img grid-0002.png --strength 0.8 --ckpt model/apple-coreml-sd-2.ckpt
No luck with this, reverted the changed in ddim in this particular directory
Another way
Going to try this method which Lieven sent me Stable diffusion on an M1 Mac
I followed the exact instructions from the Singleton blog, I did have to downgrade to a slightly older version of pytorch(lightning, i think) because the instructions do not specify specific versions of the packages which is annoying. Additionally, I think it might just be a memory issue because this really isnt optimised for apple silicion and i can’t generate 512×512 images, it only seems to work if i use 256×256. I might try and use this for now and see how i get on so i can at least have a structure for what’s going to work when i eventually get it working on a windows machine.
Supervisor discussion
Talked about recording my documentation – should talk about my research
What I want to say about my research, indicate references in slides
Can do each point of report structure as a separate take or separate presentation
Talk about the activisism element of cirio, more detail about the artworks.
Bibliography
Richards, N.M. (2013) The Dangers of Surveillance, 126 (7 ), pp. 1934–1965.