DiffusionAi/stablediffusion-infinity
1
0
Fork 0
mirror of https://github.com/lkwq007/stablediffusion-infinity.git synced 2025-01-05 10:26:59 +08:00
Code Issues Packages Projects Releases Wiki Activity
Outpainting with Stable Diffusion on an infinite canvas
253 Commits 9 Branches 3 Tags 17 MiB
Python 71.4%
JavaScript 18.5%
HTML 8%
Jupyter Notebook 1.2%
Shell 0.7%
Other 0.2%
master
Go to file
lnyan 2e53d77819 Upgrade diffuesrs
2023-01-24 12:24:57 +08:00
.github/ISSUE_TEMPLATE Update bug_report.md 2022-10-07 22:18:05 +08:00
blip_model@32eb46a646 Add blip submodule 2022-10-23 16:02:54 +08:00
css Pin w2ui css&js 2022-10-19 13:04:08 +08:00
docker fix: correct workdir owner 2022-12-04 03:37:45 +01:00
docs Update setup guide 2023-01-23 23:14:57 +08:00
glid_3_xl_stable@876ad08de2 Bump diffusers to 0.5.1 2022-10-14 19:50:13 +08:00
js Update proceed.js 2022-10-24 12:06:55 +08:00
models Update checkpoint loading 2022-10-20 22:20:17 +08:00
PyPatchMatch@0396a55825 Suppress error messages for PyPatchMatch 2023-01-21 00:35:27 +08:00
sd_grpcserver@d512d0cbc3 Add latent g_diffuser_lib from https://github.com/hafriedlander/stable-diffusion-grpcserver 2022-10-20 00:15:33 +08:00
.dockerignore fix: create dockerignore 2022-12-04 02:32:46 +01:00
.gitignore no longer need token in docker ( since the app does it ) 2022-10-04 23:50:52 -05:00
.gitmodules Add blip submodule 2022-10-23 16:02:54 +08:00
app.py Upgrade diffuesrs 2023-01-24 12:24:57 +08:00
canvas.py Update selection_size 2022-10-21 23:21:57 +08:00
config.yaml Update keyboard shortcut support 2022-10-19 22:24:21 +08:00
convert_checkpoint.py Update checkpoint loading 2022-10-20 22:20:17 +08:00
docker-compose.yml refactor: add app content inside image 2022-12-04 02:38:52 +01:00
environment.yml Upgrade diffuesrs 2023-01-24 12:24:57 +08:00
index.html Update toolbar 2022-10-23 22:52:38 +08:00
interrogate.py Update interrogate 2022-10-23 19:27:45 +08:00
LICENSE Add license 2022-09-18 11:38:50 +08:00
perlin2d.py Move code for perlin noise to a separate file 2022-09-18 11:34:03 +08:00
postprocess.py Add keyboard shortcut support 2022-10-19 22:04:14 +08:00
process.py disable some backends for fpie 2022-10-21 15:54:37 +08:00
readme.md Add clip interrogator v1 2022-10-23 14:11:08 +08:00
stablediffusion_infinity_colab.ipynb Update onnx support 2023-01-23 22:52:31 +08:00
utils.py Fix dtype issue for legacy pipeline 2022-10-23 19:20:49 +08:00

readme.md

stablediffusion-infinity

Outpainting with Stable Diffusion on an infinite canvas.

Open In Colab Hugging Face Spaces Setup Locally

outpaint

https://user-images.githubusercontent.com/1665437/197244111-51884b3b-dffe-4dcf-a82a-fa5117c79934.mp4

Status

Powered by Stable Diffusion inpainting model, this project now works well. However, the quality of results is still not guaranteed. You may need to do prompt engineering, change the size of the selection, reduce the size of the outpainting region to get better outpainting results.

The project now becomes a web app based on PyScript and Gradio. For Jupyter Notebook version, please check out the ipycanvas branch.

Pull requests are welcome for better UI control, ideas to achieve better results, or any other improvements.

Update: the project add photometric correction to suppress seams, to use this feature, you need to install fpie: pip install fpie (Linux/MacOS only)

Docs

Get Started

FAQs

  • The result is a black square:
    • False positive rate of safety checker is relatively high, you may disable the safety_checker
    • Some GPUs might not work with fp16: python app.py --fp32 --lowvram
  • What is the init_mode
    • init_mode indicates how to fill the empty/masked region, usually patch_match is better than others
  • Why not use postMessage for iframe interaction
    • The iframe and the gradio are in the same origin. For postMessage version, check out gradio-space version

Known issues

  • The canvas is implemented with NumPy + PyScript (the project was originally implemented with ipycanvas inside a jupyter notebook), which is relatively inefficient compared with pure frontend solutions.
  • By design, the canvas is infinite. However, the canvas size is finite in practice. Your RAM and browser limit the canvas size. The canvas might crash or behave strangely when zoomed out by a certain scale.
  • The canvas requires internet: You can deploy and serve PyScript, Pyodide, and other JS/CSS assets with a local HTTP server and modify index.html accordingly.
  • Photometric correction might not work (taichi does not support the multithreading environment). A dirty hack (quite unreliable) is implemented to move related computation inside a subprocess.
  • Stable Diffusion inpainting model is much slower when selection size is larger than 512x512

Credit

The code of perlin2d.py is from https://stackoverflow.com/questions/42147776/producing-2d-perlin-noise-with-numpy/42154921#42154921 and is not included in the scope of LICENSE used in this repo.

The submodule glid_3_xl_stable is based on https://github.com/Jack000/glid-3-xl-stable

The submodule PyPatchMatch is based on https://github.com/vacancy/PyPatchMatch

The code of postprocess.py and process.py is modified based on https://github.com/Trinkle23897/Fast-Poisson-Image-Editing

The code of convert_checkpoint.py is modified based on https://github.com/huggingface/diffusers/blob/main/scripts/convert_original_stable_diffusion_to_diffusers.py

The submodule sd_grpcserver and handleImageAdjustment() in utils.py are based on https://github.com/hafriedlander/stable-diffusion-grpcserver and https://github.com/parlance-zz/g-diffuser-bot

w2ui.min.js and w2ui.min.css is from https://github.com/vitmalina/w2ui. fabric.min.js is a custom build of https://github.com/fabricjs/fabric.js

interrogate.py is based on https://github.com/pharmapsychotic/clip-interrogator v1, the submodule blip_model is based on https://github.com/salesforce/BLIP