[bug-fix] Fix reinstall-overwrites-kept-config: preserve config on plain reinstall after --keep-config#3449
Open
github-actions[bot] wants to merge 20 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes extension config loss when reinstalling after remove --keep-config.
Changes:
- Rescues and restores preserved extension configuration files.
- Adds regression tests for standard and local configs.
Show a summary per file
| File | Description |
|---|---|
src/specify_cli/extensions/__init__.py |
Adds preserved-config rescue and restoration. |
tests/test_extensions.py |
Adds reinstall config-preservation tests. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 3
- Review effort level: Medium
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
…rt' and 'import from'' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
- Stage stranded config files to a durable rescue_staging_dir (extensions_dir/.rescue-staging-<id>) before rmtree so original bytes survive partial rmtree, copytree failure, or partial restore on retry. On retry the staging dir is detected and its content reused instead of whatever mix of packaged defaults and partial restores remains on disk. The staging dir is cleaned up only after every restore succeeds. - Fix CodeQL: change `import specify_cli.extensions as _ext_module` to `from specify_cli import extensions as _ext_module` in test file. Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
…up errors - Thread 14: Change except BaseException to except Exception in the staging fallback block so KeyboardInterrupt/SystemExit propagate correctly - Thread 15: Add explanatory comment to the bare pass in the chmod except block to satisfy static analysis - Thread 16: Reject a symlinked staging directory and only reload non-symlinked files whose names match the two recognised config suffixes - Thread 17: Create each staging file via os.open with mode 0600 and O_CREAT|O_EXCL before writing so preserved bytes are never transiently exposed to other local users - Thread 18: Remove ignore_errors=True from the final staging-dir cleanup so a failed rmtree propagates rather than silently leaving a stale backup that could be misread on the next retry Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
- Add `import stat` to imports - Use `stat.S_IMODE(mode)` before chmod in staging write (thread 20, line 1464) - Use `stat.S_IMODE(preserved_mode)` and make chmod best-effort in `_restore_stranded_config_file` (thread 18, line 1492) - Add `not rescue_staging_dir.is_symlink()` guard to cleanup (thread 19, line 1522) Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
…re, full os.write Assisted-by: GitHub Copilot (model: GPT-5.3-Codex, autonomous)
Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
mnriem
force-pushed
the
fix/3427-reinstall-overwrites-kept-config-40c76b4f5dd346d6
branch
from
6ff00ec to
86f93b6
Compare
Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…rt' and 'import from'' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Comment on lines
+117
to
+119
| os.fsync(dir_fd) | ||
| except (AttributeError, OSError, NotImplementedError): | ||
| pass |
Comment on lines
+1518
to
+1521
| try: | ||
| os.fsync(fd) | ||
| except (AttributeError, OSError, NotImplementedError): | ||
| pass |
Comment on lines
+1597
to
+1601
| # Every stranded config has been restored successfully — the | ||
| # durable staging backup is no longer needed. Raise on failure so | ||
| # the install is not reported as successful while a stale backup | ||
| # that could be misread on the next retry remains on disk. | ||
| if rescue_staging_dir.is_dir() and not rescue_staging_dir.is_symlink(): |
Comment on lines
+1407
to
+1413
| assert staging_dir.exists() | ||
| assert (staging_dir / ".rescue-complete").exists() | ||
| assert (staging_dir / "test-ext-config.yml").exists() | ||
|
|
||
| manifest = manager.install_from_directory( | ||
| extension_dir, "0.1.0", register_commands=False | ||
| ) |
Comment on lines
+1668
to
+1669
| "[yellow]Warning:[/yellow] Could not remove temporary " | ||
| f"download file {_escape_markup(str(tmp_path))}: " | ||
| f"{_escape_markup(str(cleanup_exc))}" | ||
| f"workflow download file: {_escape_markup(str(cleanup_exc))}" |
Comment on lines
+1692
to
+1693
| "[yellow]Warning:[/yellow] Could not remove temporary " | ||
| f"download file {_escape_markup(str(tmp_path))}: " | ||
| f"{_escape_markup(str(exc))}" | ||
| f"workflow download file: {_escape_markup(str(exc))}" |
Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
| if target_fd is not None: | ||
| try: | ||
| os.close(target_fd) | ||
| except OSError: |
| from unittest.mock import MagicMock | ||
|
|
||
| from tests.conftest import strip_ansi | ||
| import specify_cli.extensions as _ext_module |
Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Comment on lines
+122
to
+123
| if not path.exists(): | ||
| return |
Comment on lines
+1546
to
+1548
| # Write the completion marker only after every staged file is | ||
| # fully written so a retry trusts staging only when it is whole. | ||
| marker_fd = os.open( |
| "[yellow]Warning:[/yellow] Could not remove temporary " | ||
| f"download file {_escape_markup(str(tmp_path))}: " | ||
| f"{_escape_markup(str(cleanup_exc))}" | ||
| f"workflow download file: {_escape_markup(str(cleanup_exc))}" |
| "[yellow]Warning:[/yellow] Could not remove temporary " | ||
| f"download file {_escape_markup(str(tmp_path))}: " | ||
| f"{_escape_markup(str(exc))}" | ||
| f"workflow download file: {_escape_markup(str(exc))}" |
Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug fix — reinstall-overwrites-kept-config
Proposed fix for issue #3427, applying the remediation from the bug assessment.
Verdict: Valid · Severity: medium
Summary
When
specify extension remove <ext> --keep-configis used, the extension is unregistered but its*-config.ymlfiles survive in the extension directory. A subsequent plainspecify extension add <ext>unconditionally deleted that directory before copying the fresh extension in, silently discarding the preserved config. The fix rescues those stranded config files into memory before thermtreeand writes them back aftercopytree, so user-customized values always win over the packaged defaults.Changes
src/specify_cli/extensions/__init__.pyrmtree(dest_dir), collect any*-config.yml/*-config.local.ymlfiles from an unregistereddest_dirinto memory; restore them aftercopytreetests/test_extensions.pytest_reinstall_after_keep_config_preserves_configandtest_reinstall_after_keep_config_preserves_local_configTests Added or Updated
tests/test_extensions.py::TestInstallFromDirectory::test_reinstall_after_keep_config_preserves_config— pins that a customized*-config.ymlsurvives aremove --keep-config→ plain reinstall cycletests/test_extensions.py::TestInstallFromDirectory::test_reinstall_after_keep_config_preserves_local_config— same for*-config.local.ymloverride filesLocal Verification
--forcereinstall path, applying it to the previously-unhandled "unregistered but config-bearing directory" case.Deviations from Assessment
None. The implementation follows the preferred remediation exactly as described.
Risks & Review Notes
not self.registry.is_installed(manifest.id)guard ensures we only rescue configs when the extension is genuinely unregistered, avoiding picking up stale files from a different install.shutil.copytree, so packaged defaults are always superseded by the user's values — no risk of defaults silently winning.install_from_zip()delegates toinstall_from_directory(), so it is covered without additional changes.install_from_directory().Refs #3427 · cc
@grafvonbremove --keep-config#3427