Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

During parallel multi-file compilation, kclvm may never stop after panic. #82

Closed
zong-zhe opened this issue Jun 8, 2022 · 0 comments · Fixed by #71
Closed

During parallel multi-file compilation, kclvm may never stop after panic. #82

zong-zhe opened this issue Jun 8, 2022 · 0 comments · Fixed by #71
Assignees
@zong-zhe
Labels
bug Something isn't working error-handling Issues or PRs related to kcl error handling invalid This doesn't seem right
Milestone
v0.4.3 Release

Comments

@zong-zhe
Copy link
Contributor

zong-zhe commented Jun 8, 2022

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

Parallel multi-file compilation code looks like follows:

1. let pool = ThreadPool::new(self.thread_count);
2. let (tx, rx) = channel();
3. for .... {
4.   let tx = tx.clone();
5.    pool.execute(move || {
6.        ...
7.    tx.send(dylib_path).expect("channel will be there waiting for the pool");
8.    });
9. }
10. rx.iter().take(prog_count).collect::<Vec<String>>()

If panic occurs when executing lines 5 to 8, the corresponding thread will be killed immediately, then "tx" in line 7 will not send the result, and "rx" in line 10 will wait for the result from "tx" and the main thread will never stop .

2. What did you expect to see? (Required)

The program outputs a normal error message and stops.

3. What did you see instead (Required)

The program cannot stop after outputting panic information.

4. What is your KusionStack components version? (Required)

kclvm version is 0.4.2; checksum: e07ed7af0d9bd1e86a3131714e4bd20c
@zong-zhe zong-zhe added bug Something isn't working invalid This doesn't seem right error-handling Issues or PRs related to kcl error handling labels Jun 8, 2022
@zong-zhe zong-zhe added this to the v0.4.3 Release milestone Jun 8, 2022
@zong-zhe zong-zhe self-assigned this Jun 8, 2022
@Peefy Peefy mentioned this issue Jun 20, 2022
Merged
15 tasks
zong-zhe pushed a commit that referenced this issue Jul 8, 2022
refactor(kclvm-runner): encapsulate lib generating, linking and execu…
debf405 
…ting into kclvm-runner.

1. Encapsulate generating of libs  in "kclvm/src/lib.rs" and "kclvm/src/main.rs"  into "kclvm/runner/assembler.rs".
2. Encapsulate linking of libs in "kclvm/src/lib.rs" and "kclvm/src/main.rs" into "kclvm/runner/linker.rs"
3. Encapsulate executing of libs  in "kclvm/src/lib.rs" and "kclvm/src/main.rs" into "kclvm/runner/lib.rs".
4. A timer is added during the concurrent multi-file compilation to prevent KCLVM locked due to child thread panic.

fix #67 #106 #82
@zong-zhe zong-zhe linked a pull request Jul 8, 2022 that will close this issue
refactor(kclvm-runner): encapsulate dylib generating, linking and executing in kclvm/lib.rs into kclvm-runner #71
Merged
15 tasks
zong-zhe pushed a commit that referenced this issue Jul 11, 2022
refactor(kclvm-runner): encapsulate lib generating, linking and execu…
6409a6d 
…ting into kclvm-runner.

1. Encapsulate generating of libs  in "kclvm/src/lib.rs" and "kclvm/src/main.rs"  into "kclvm/runner/assembler.rs".
2. Encapsulate linking of libs in "kclvm/src/lib.rs" and "kclvm/src/main.rs" into "kclvm/runner/linker.rs"
3. Encapsulate executing of libs  in "kclvm/src/lib.rs" and "kclvm/src/main.rs" into "kclvm/runner/lib.rs".
4. A timer is added during the concurrent multi-file compilation to prevent KCLVM locked due to child thread panic.

fix #67 #106 #82
@Peefy Peefy closed this as completed in #71 Jul 11, 2022
Peefy added a commit that referenced this issue Jul 11, 2022
@zong-zhe @Peefy
refactor(kclvm-runner): encapsulate dylib generating, linking and exe…
6f7ec98 
…cuting in kclvm/lib.rs into kclvm-runner (#71)

* refactor(kclvm-runner): encapsulate dylib generating, linking and executing in kclvm/lib.rs into kclvm-runner

The assembler and linker of the current version of the compiler are not separately packaged.
In order to support the reuse of modules such as dylibs generating，linking and executing,
This modification separates dylibs generating，and encapsulate them individually into KclvmAssembler and KclvmLinker.

Encapsulate dylibs generating, linking and executing into kclvm-runner/assembler.rs and kclvm-runner/linker.rs.
Add struct "KclvmAssembler" in kclvm-runner/assembler.rs to provide method "gen_dylibs" for dylibs generating.
Add struct "KclvmLinker" in kclvm-runner/linker.rs to provide method "link_all_dylibs" for dylib linking.
Add method "execute" in kclvm-runner/lib.rs to encapsulate dylibs generating(gen_dylib), dylib linking(link_all_dylib) and running(runner.run) together.

fix #67

* chore:  bump plugins submodule to 23fc581d (#64)

* refactor(kclvm-runner): merge main and refacor kclvm-runner.

                    1. Encapsulated method "emit_code" into "lock_ll_file_and_gen_dylib" to
                       reduce repetitive code in kclvm-runner/KclvmAssembler.gen_dylibs().
                    2. In order to support reuse and simplify the structure of Method "gen_dylibs()",
                       encapsulates some operations of cleaning file paths.
                    3. Due to issue #79, some test cases are temporarily commented out
                    2. In order to support reuse and simplify the structure of Method "gen_dylibs()",
                       encapsulates some operations of cleaning file paths.
                    3. Due to issue #79, some test cases are temporarily commented out

                    fix #67

                    refactor(kclvm-runner): decouping assembler and llvm.

                    1. Decoupling the assembler and llvm.
                    2. The assembling LLVM IR into a dynamic link library is encapsulated into "LlvmLibAssembler" separately.
                    3. Add trait "LibAssembler" to describe the interface "KclvmLibAssembler" should have.
                       If other intermediate code is added in the future, the corresponding assembler must implement this trait.
                    4. Struct "LlvmLibAssembler" is provided in "KclvmLibAssembler".
                       "KclvmLibAssembler" is an enum that implements trait "LibAssembler".
                       "KclvmLibAssembler" is responsible for the compilation of a single kcl file in one thread.
                    , 5. "KclvmAssembler" is responsible for the concurrent compilation of multiple kcl files.
                    6. "KclvmAssembler" will call the method in "KclvmLibAssembler" to generate a dynamic link library
                       for a single kcl file in each thread of concurrent compilation.

                    fix #67

* Merge branch 'main' into refactor/zong-zhe/add_eval_to_kclvm_runner

* add kclvm path in ci for ubuntu

* refactor(kclvm-runner): encapsulate lib generating, linking and executing into kclvm-runner.

1. Encapsulate generating of libs  in "kclvm/src/lib.rs" and "kclvm/src/main.rs"  into "kclvm/runner/assembler.rs".
2. Encapsulate linking of libs in "kclvm/src/lib.rs" and "kclvm/src/main.rs" into "kclvm/runner/linker.rs"
3. Encapsulate executing of libs  in "kclvm/src/lib.rs" and "kclvm/src/main.rs" into "kclvm/runner/lib.rs".
4. A timer is added during the concurrent multi-file compilation to prevent KCLVM locked due to child thread panic.

fix #67 #106 #82

Co-authored-by: Peefy <[email protected]>
eminaktas pushed a commit to eminaktas/kcl that referenced this issue Oct 17, 2024
@amyXia1994
fix Server model; add redis components for guestbook app (kcl-lang#82)
11620a1 
* add name attribute on Server model; add redis components for guestbook app

* update server model: remove boilerplate

* update server label key
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zong-zhe zong-zhe

Labels
bug Something isn't working error-handling Issues or PRs related to kcl error handling invalid This doesn't seem right
Projects
None yet
Milestone
v0.4.3 Release
1 participant
@zong-zhe