I was trying to try out some parallelism in my code with RF. I'm running into a potential issue, I was wondering if it was a bug.

The following two simple examples for multiprocessing are found on this page: https://realpython.com/python-concurrency/

import multiprocessing
import time


def cpu_bound(number):
    return sum(i * i for i in range(number))


def find_sums(numbers):
    with multiprocessing.Pool() as pool:
        pool.map(cpu_bound, numbers)


if __name__ == "__main__":
    numbers = [5_000_000 + x for x in range(20)]

    start_time = time.time()
    find_sums(numbers)
    duration = time.time() - start_time
    print(f"Duration {duration} seconds")

import requests
import multiprocessing
import time

session = None


def set_global_session():
    global session
    if not session:
        session = requests.Session()


def download_site(url):
    with session.get(url) as response:
        name = multiprocessing.current_process().name
        print(f"{name}:Read {len(response.content)} from {url}")


def download_all_sites(sites):
    with multiprocessing.Pool(initializer=set_global_session) as pool:
        pool.map(download_site, sites)


if __name__ == "__main__":
    sites = [
        "https://www.jython.org",
        "http://olympus.realpython.org/dice",
    ] * 80
    start_time = time.time()
    download_all_sites(sites)
    duration = time.time() - start_time
    print(f"Downloaded {len(sites)} in {duration} seconds")

For both examples, the method that is called by the multiprocessing pool (cpu_bound in the first and download_site in the second) are met with the same PicklingError:

Traceback (most recent call last):
  File "<untitled>", line 18, in <module>
  File "<untitled>", line 11, in find_sums
  File "multiprocessing/pool.pyc", line 268, in map
  File "multiprocessing/pool.pyc", line 657, in get
  File "multiprocessing/pool.pyc", line 431, in _handle_tasks
  File "multiprocessing/connection.pyc", line 206, in send
  File "multiprocessing/reduction.pyc", line 51, in dumps
_pickle.PicklingError: Can't pickle <function cpu_bound at 0x148c59320>: attribute lookup cpu_bound on __main__ failed

Both examples run in terminal and in my IDE A-OK.

I was wondering if either of you would know what was going on here. Is there a way I should edit the examples above to get them to run, or is it a bug in RF?

Thanks!

asyncio examples does work... I assume, without doing to much research, the issue is related to the combo with pyObject and an already running runloop. RoboFont is one big runloop.

Essentially where I was hoping to go with this is I think it would be great to somehow run a process in the background without RoboFont locking up the interface. Even this simple asyncio example, while it does execute in parallel, causes RoboFont to "beachball" while the process completes.

import asyncio

async def count():
    print("One")
    await asyncio.sleep(10)
    print("Two")

async def main():
    await asyncio.gather(count(), count(), count())

if __name__ == "__main__":
    import time
    s = time.perf_counter()
    asyncio.run(main())
    elapsed = time.perf_counter() - s
    print(f"{__file__} executed in {elapsed:0.2f} seconds.")

Is there any way to spawn a process or run a script, maybe on another core (I have 8 cores on this iMac, RoboFont only runs on 1), that runs in the background, and doesn't block RF's other processes?

I tried it in DrawBot because I seem to remember trying multiprocessing a year ago for a drawbot thing after seeing this post https://forum.drawbot.com/topic/150/drawbot-multiprocessing?_=1591994686928

and it worked back then! I wonder what changed?

Threading, multiprocessing is harder with user interaction and with the bridge between python and cocoa...

FontGoggles uses https://github.com/alberthier/corefoundationasyncio

in RF and DB the main module must be the py file starting the app!

import sys

print(sys.modules["__main__"])

while when you run your code in terminal the __main__ module is the file itself...

do an import of the your multiprocessing code instead.

this allows to create a task in a run loop:

# will not work in RF3.4 or DB1.126 but will work in the next releases!! 
import asyncio

async def count():
    print("One")
    await asyncio.sleep(10)
    print("Two")

async def main():
    await asyncio.gather(count(), count(), count())

if __name__ == "__main__":
    asyncio.create_task(main())

Importing the multiprocessing code is indeed the trick; the demos work now perfectly.

And I can't wait to try out the new async stuff in the next releases!

Thanks!!