[med-svn] [Git][med-team/conda-package-streaming][upstream] 2 commits: New upstream version 0.9.0
Andreas Tille (@tille)
gitlab at salsa.debian.org
Tue Jun 25 19:41:42 BST 2024
Andreas Tille pushed to branch upstream at Debian Med / conda-package-streaming
Commits:
1a9d1a7e by Andreas Tille at 2023-09-11T12:29:53+02:00
New upstream version 0.9.0
- - - - -
1be1cd5c by Andreas Tille at 2024-06-25T20:38:50+02:00
New upstream version 0.10.0
- - - - -
21 changed files:
- + .github/ISSUE_TEMPLATE/0_bug.yml
- + .github/ISSUE_TEMPLATE/1_feature.yml
- + .github/ISSUE_TEMPLATE/2_documentation.yml
- + .github/ISSUE_TEMPLATE/epic.yml
- + .github/workflows/cla.yml
- + .github/workflows/issues.yml
- + .github/workflows/labels.yml
- + .github/workflows/lock.yml
- + .github/workflows/project.yml
- + .github/workflows/stale.yml
- .pre-commit-config.yaml
- CHANGELOG.md
- CODE_OF_CONDUCT.md
- + HOW_WE_USE_GITHUB.md
- conda_package_streaming/__init__.py
- conda_package_streaming/package_streaming.py
- conda_package_streaming/transmute.py
- tests/server.py
- tests/test_extract.py
- tests/test_streaming.py
- tests/test_transmute.py
Changes:
=====================================
.github/ISSUE_TEMPLATE/0_bug.yml
=====================================
@@ -0,0 +1,77 @@
+name: Bug Report
+description: Create a bug report.
+labels:
+ - type::bug
+body:
+ - type: markdown
+ attributes:
+ value: |
+ Because processing new bug reports is time-consuming, we would like to ask you to fill out the following form to the best of your ability and as completely as possible.
+
+ > [!NOTE]
+ > Bug reports that are incomplete or missing information may be closed as inactionable.
+
+ Since there are already a lot of open issues, please also take a moment to search existing ones to see if your bug has already been reported. If you find something related, please upvote that issue and provide additional details as necessary.
+
+ 💐 Thank you for helping to make Conda better. We would be unable to improve Conda without our community!
+ - type: checkboxes
+ id: checks
+ attributes:
+ label: Checklist
+ description: Please confirm and check all of the following options.
+ options:
+ - label: I added a descriptive title
+ required: true
+ - label: I searched open reports and couldn't find a duplicate
+ required: true
+ - type: textarea
+ id: what
+ attributes:
+ label: What happened?
+ description: What should have happened instead? Please provide as many details as possible. The more information provided, the more likely we are able to replicate your problem and offer a solution.
+ validations:
+ required: true
+ - type: textarea
+ id: info
+ attributes:
+ label: Conda Info
+ description: |
+ Let's collect some basic information about your conda install.
+
+ Please run the following command from your command line and paste the output below.
+
+ ```bash
+ conda info
+ ```
+ render: shell
+ - type: textarea
+ id: config
+ attributes:
+ label: Conda Config
+ description: |
+ Let's collect any customizations you may have for your conda install.
+
+ Please run the following command from your command line and paste the output below.
+
+ ```bash
+ conda config --show-sources
+ ```
+ render: shell
+ - type: textarea
+ id: list
+ attributes:
+ label: Conda list
+ description: |
+ The packages installed into your environment can offer clues as to the problem you are facing.
+
+ Please activate the environment within which you are encountering this bug, run the following command from your command line, and paste the output below.
+
+ ```bash
+ conda list --show-channel-urls
+ ```
+ render: shell
+ - type: textarea
+ id: context
+ attributes:
+ label: Additional Context
+ description: Include any additional information (or screenshots) that you think would be valuable.
=====================================
.github/ISSUE_TEMPLATE/1_feature.yml
=====================================
@@ -0,0 +1,48 @@
+name: Feature Request
+description: Create a feature request.
+labels:
+ - type::feature
+body:
+ - type: markdown
+ attributes:
+ value: |
+ Because processing new feature requests is time-consuming, we would like to ask you to fill out the following form to the best of your ability and as completely as possible.
+
+ > [!NOTE]
+ > Feature requests that are incomplete or missing information may be closed as inactionable.
+
+ Since there are already a lot of open issues, please also take a moment to search existing ones to see if your feature request has already been submitted. If you find something related, please upvote that issue and provide additional details as necessary.
+
+ 💐 Thank you for helping to make Conda better. We would be unable to improve Conda without our community!
+ - type: checkboxes
+ id: checks
+ attributes:
+ label: Checklist
+ description: Please confirm and check all of the following options.
+ options:
+ - label: I added a descriptive title
+ required: true
+ - label: I searched open requests and couldn't find a duplicate
+ required: true
+ - type: textarea
+ id: idea
+ attributes:
+ label: What is the idea?
+ description: Describe what the feature is and the desired state.
+ validations:
+ required: true
+ - type: textarea
+ id: why
+ attributes:
+ label: Why is this needed?
+ description: Who would benefit from this feature? Why would this add value to them? What problem does this solve?
+ - type: textarea
+ id: what
+ attributes:
+ label: What should happen?
+ description: What should be the user experience with the feature? Describe from a user perspective what they would do and see.
+ - type: textarea
+ id: context
+ attributes:
+ label: Additional Context
+ description: Include any additional information that you think would be valuable.
=====================================
.github/ISSUE_TEMPLATE/2_documentation.yml
=====================================
@@ -0,0 +1,36 @@
+name: Documentation
+description: Create a documentation related issue.
+labels:
+ - type::documentation
+body:
+ - type: markdown
+ attributes:
+ value: |
+ > [!NOTE]
+ > Documentation requests that are incomplete or missing information may be closed as inactionable.
+
+ Since there are already a lot of open issues, please also take a moment to search existing ones to see if your bug has already been reported. If you find something related, please upvote that issue and provide additional details as necessary.
+
+ 💐 Thank you for helping to make conda better. We would be unable to improve conda without our community!
+ - type: checkboxes
+ id: checks
+ attributes:
+ label: Checklist
+ description: Please confirm and check all of the following options.
+ options:
+ - label: I added a descriptive title
+ required: true
+ - label: I searched open reports and couldn't find a duplicate
+ required: true
+ - type: textarea
+ id: what
+ attributes:
+ label: What happened?
+ description: Mention here any typos, broken links, or missing, incomplete, or outdated information, etc. that you have noticed in the conda docs or CLI help.
+ validations:
+ required: true
+ - type: textarea
+ id: context
+ attributes:
+ label: Additional Context
+ description: Include any additional information (or screenshots) that you think would be valuable.
=====================================
.github/ISSUE_TEMPLATE/epic.yml
=====================================
@@ -0,0 +1,105 @@
+name: Epic
+description: A collection of related tickets.
+labels:
+ - epic
+body:
+ - type: markdown
+ attributes:
+ value: |
+ This form is intended for grouping and collecting together related tickets to better gauge the scope of a problem/feature.
+
+ If you are attempting to report a bug, propose a new feature, or some other code change please use one of the other forms available.
+
+ > [!NOTE]
+ > Epics that are incomplete or missing information may be closed as inactionable.
+
+ Since there are already a lot of open issues, please also take a moment to search existing ones to see if a similar epic has already been opened. If you find something related, please upvote that issue and provide additional details as necessary.
+
+ 💐 Thank you for helping to make Conda better. We would be unable to improve Conda without our community!
+ - type: checkboxes
+ id: checks
+ attributes:
+ label: Checklist
+ description: Please confirm and check all of the following options.
+ options:
+ - label: I added a descriptive title
+ required: true
+ - label: I searched open issues and couldn't find a duplicate
+ required: true
+
+ - type: textarea
+ id: what
+ attributes:
+ label: What?
+ description: >-
+ What feature or problem will be addressed in this epic?
+ placeholder: Please describe here.
+ validations:
+ required: true
+
+ - type: textarea
+ id: why
+ attributes:
+ label: Why?
+ description: >-
+ Why is the reported issue(s) a problem, or why is the proposed feature needed?
+ (Research and spike issues can be linked here.)
+ value: |
+ - [ ] <supporting issue from user, research, or spike>
+ placeholder: Please describe here and/or link to relevant supporting issues.
+ validations:
+ required: true
+
+ - type: textarea
+ id: user_impact
+ attributes:
+ label: User impact
+ description: >-
+ In what specific way(s) will users benefit from this change? (e.g. use cases or performance improvements)
+ placeholder: Please describe here.
+ validations:
+ required: true
+
+ - type: textarea
+ id: goals
+ attributes:
+ label: Goals
+ description: >-
+ What goal(s) should this epic accomplish?
+ value: |
+ - [ ] <goal>
+ validations:
+ required: true
+
+ - type: textarea
+ id: tasks
+ attributes:
+ label: Tasks
+ description: >-
+ What needs to be done to implement this change?
+ value: |
+ - [ ] <task or PR>
+ validations:
+ required: false
+
+ - type: textarea
+ id: blocked_by
+ attributes:
+ label: 'This epic is blocked by:'
+ description: >-
+ Epics and issues that block this epic.
+ value: |
+ - [ ] <dependency epic or issue>
+ validations:
+ required: false
+
+ - type: textarea
+ id: blocks
+ attributes:
+ label: 'This epic blocks:'
+ description: >-
+ Epics and issues that are blocked by this epic.
+ value: |
+ - [ ] <dependency epic or issue>
+ validations:
+ required: false
=====================================
.github/workflows/cla.yml
=====================================
@@ -0,0 +1,35 @@
+name: CLA
+
+on:
+ issue_comment:
+ types:
+ - created
+ pull_request_target:
+
+jobs:
+ check:
+ if: >-
+ !github.event.repository.fork
+ && (
+ github.event.issue.pull_request
+ && github.event.comment.body == '@conda-bot check'
+ || github.event_name == 'pull_request_target'
+ )
+ runs-on: ubuntu-latest
+ steps:
+ - name: Check CLA
+ uses: conda/actions/check-cla at 976289d0cfd85139701b26ddd133abdd025a7b5f # v24.5.0
+ with:
+ # [required]
+ # A token with ability to comment, label, and modify the commit status
+ # (`pull_request: write` and `statuses: write` for fine-grained PAT; `repo` for classic PAT)
+ # (default: secrets.GITHUB_TOKEN)
+ token: ${{ secrets.CLA_ACTION_TOKEN }}
+ # [required]
+ # Label to apply to contributor's PR once CLA is signed
+ label: cla-signed
+
+ # [required]
+ # Token for opening signee PR in the provided `cla_repo`
+ # (`pull_request: write` for fine-grained PAT; `repo` and `workflow` for classic PAT)
+ cla_token: ${{ secrets.CLA_FORK_TOKEN }}
=====================================
.github/workflows/issues.yml
=====================================
@@ -0,0 +1,35 @@
+name: Automate Issues
+
+on:
+ # NOTE: github.event is issue_comment payload:
+ # https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#issue_comment
+ issue_comment:
+ types: [created]
+
+env:
+ FEEDBACK_LBL: pending::feedback
+ SUPPORT_LBL: pending::support
+
+jobs:
+ # NOTE: will update label if anyone responds, not just the author/reporter
+ # TODO: create conda-issue-sorting team and modify this to toggle label based on
+ # whether a non-issue-sorting engineer commented
+ pending_support:
+ # if [pending::feedback] and anyone responds
+ if: >-
+ !github.event.repository.fork
+ && !github.event.issue.pull_request
+ && contains(github.event.issue.labels.*.name, 'pending::feedback')
+ runs-on: ubuntu-latest
+ steps:
+ # remove [pending::feedback]
+ - uses: actions-ecosystem/action-remove-labels at 2ce5d41b4b6aa8503e285553f75ed56e0a40bae0 # v1.3.0
+ with:
+ labels: ${{ env.FEEDBACK_LBL }}
+ github_token: ${{ secrets.PROJECT_TOKEN }}
+ # add [pending::support], if still open
+ - uses: actions-ecosystem/action-add-labels at 18f1af5e3544586314bbe15c0273249c770b2daf # v1.1.3
+ if: github.event.issue.state == 'open'
+ with:
+ labels: ${{ env.SUPPORT_LBL }}
+ github_token: ${{ secrets.PROJECT_TOKEN }}
=====================================
.github/workflows/labels.yml
=====================================
@@ -0,0 +1,42 @@
+name: Sync Labels
+
+on:
+ # NOTE: github.event is workflow_dispatch payload:
+ # https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#workflow_dispatch
+ workflow_dispatch:
+ inputs:
+ dryrun:
+ description: 'dryrun: Preview changes to labels without editing them (true|false)'
+ required: true
+ type: boolean
+ default: true
+
+jobs:
+ sync:
+ if: '!github.event.repository.fork'
+ runs-on: ubuntu-latest
+ env:
+ GLOBAL: https://raw.githubusercontent.com/conda/infra/main/.github/global.yml
+ LOCAL: .github/labels.yml
+ steps:
+ - uses: actions/checkout at a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
+ - id: has_local
+ uses: andstor/file-existence-action at 076e0072799f4942c8bc574a82233e1e4d13e9d6 # v3.0.0
+ with:
+ files: ${{ env.LOCAL }}
+ - name: Global Only
+ uses: EndBug/label-sync at 52074158190acb45f3077f9099fea818aa43f97a # v2.3.3
+ if: steps.has_local.outputs.files_exists == 'false'
+ with:
+ config-file: ${{ env.GLOBAL }}
+ delete-other-labels: true
+ dry-run: ${{ github.event.inputs.dryrun }}
+ - name: Global & Local
+ uses: EndBug/label-sync at 52074158190acb45f3077f9099fea818aa43f97a # v2.3.3
+ if: steps.has_local.outputs.files_exists == 'true'
+ with:
+ config-file: |
+ ${{ env.GLOBAL }}
+ ${{ env.LOCAL }}
+ delete-other-labels: true
+ dry-run: ${{ github.event.inputs.dryrun }}
=====================================
.github/workflows/lock.yml
=====================================
@@ -0,0 +1,45 @@
+name: Lock
+
+on:
+ # NOTE: github.event is workflow_dispatch payload:
+ # https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#workflow_dispatch
+ workflow_dispatch:
+
+ schedule:
+ - cron: 0 6 * * *
+
+permissions:
+ issues: write
+ pull-requests: write
+
+jobs:
+ lock:
+ if: '!github.event.repository.fork'
+ runs-on: ubuntu-latest
+ steps:
+ - uses: dessant/lock-threads at 1bf7ec25051fe7c00bdd17e6a7cf3d7bfb7dc771 # v5.0.1
+ with:
+ # Number of days of inactivity before a closed issue is locked
+ issue-inactive-days: 365
+ # Do not lock issues created before a given timestamp, value must follow ISO 8601
+ exclude-issue-created-before: ''
+ # Do not lock issues with these labels, value must be a comma separated list of labels or ''
+ exclude-any-issue-labels: ''
+ # Labels to add before locking an issue, value must be a comma separated list of labels or ''
+ add-issue-labels: locked
+ # Reason for locking an issue, value must be one of resolved, off-topic, too heated, spam or ''
+ issue-lock-reason: resolved
+
+ # Number of days of inactivity before a closed pull request is locked
+ pr-inactive-days: 365
+ # Do not lock pull requests created before a given timestamp, value must follow ISO 8601
+ exclude-pr-created-before: ''
+ # Do not lock pull requests with these labels, value must be a comma separated list of labels or ''
+ exclude-any-pr-labels: ''
+ # Labels to add before locking a pull request, value must be a comma separated list of labels or ''
+ add-pr-labels: locked
+ # Reason for locking a pull request, value must be one of resolved, off-topic, too heated, spam or ''
+ pr-lock-reason: resolved
+
+ # Limit locking to issues, pull requests or discussions, value must be a comma separated list of issues, prs, discussions or ''
+ process-only: issues, prs
=====================================
.github/workflows/project.yml
=====================================
@@ -0,0 +1,21 @@
+name: Add to Project
+
+on:
+ issues:
+ types:
+ - opened
+ pull_request_target:
+ types:
+ - opened
+
+jobs:
+ add_to_project:
+ if: '!github.event.repository.fork'
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/add-to-project at 9bfe908f2eaa7ba10340b31e314148fcfe6a2458 # v1.0.1
+ with:
+ # issues are added to the Planning project
+ # PRs are added to the Review project
+ project-url: https://github.com/orgs/conda/projects/${{ github.event_name == 'issues' && 2 || 16 }}
+ github-token: ${{ secrets.PROJECT_TOKEN }}
=====================================
.github/workflows/stale.yml
=====================================
@@ -0,0 +1,98 @@
+name: Stale
+
+on:
+ # NOTE: github.event is workflow_dispatch payload:
+ # https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#workflow_dispatch
+ workflow_dispatch:
+ inputs:
+ dryrun:
+ description: 'dryrun: Preview stale issues/prs without marking them (true|false)'
+ required: true
+ type: boolean
+ default: true
+
+ schedule:
+ - cron: 0 4 * * *
+
+permissions:
+ issues: write
+ pull-requests: write
+
+jobs:
+ stale:
+ if: '!github.event.repository.fork'
+ runs-on: ubuntu-latest
+ strategy:
+ matrix:
+ include:
+ - only-issue-labels: ''
+ days-before-issue-stale: 365
+ days-before-issue-close: 30
+ # [type::support] issues have a more aggressive stale/close timeline
+ - only-issue-labels: type::support
+ days-before-issue-stale: 90
+ days-before-issue-close: 21
+ steps:
+ - uses: conda/actions/read-yaml at 976289d0cfd85139701b26ddd133abdd025a7b5f # v24.5.0
+ id: read_yaml
+ with:
+ path: https://raw.githubusercontent.com/conda/infra/main/.github/messages.yml
+
+ - uses: actions/stale at 28ca1036281a5e5922ead5184a1bbf96e5fc984e # v9.0.0
+ id: stale
+ with:
+ # Only issues with these labels are checked whether they are stale
+ only-issue-labels: ${{ matrix.only-issue-labels }}
+
+ # Idle number of days before marking issues stale
+ days-before-issue-stale: ${{ matrix.days-before-issue-stale }}
+ # Idle number of days before closing stale issues/PRs
+ days-before-issue-close: ${{ matrix.days-before-issue-close }}
+ # Idle number of days before marking PRs stale
+ days-before-pr-stale: 365
+ # Idle number of days before closing stale PRs
+ days-before-pr-close: 30
+
+ # Comment on the staled issues
+ stale-issue-message: ${{ fromJSON(steps.read_yaml.outputs.value)['stale-issue'] }}
+ # Label to apply on staled issues
+ stale-issue-label: stale
+ # Label to apply on closed issues
+ close-issue-label: stale::closed
+ # Reason to use when closing issues
+ close-issue-reason: not_planned
+
+ # Comment on the staled PRs
+ stale-pr-message: ${{ fromJSON(steps.read_yaml.outputs.value)['stale-pr'] }}
+ # Label to apply on staled PRs
+ stale-pr-label: stale
+ # Label to apply on closed PRs
+ close-pr-label: stale::closed
+
+ # Remove stale label from issues/PRs on updates/comments
+ remove-stale-when-updated: true
+ # Add specified labels to issues/PRs when they become unstale
+ labels-to-add-when-unstale: stale::recovered
+ # Remove specified labels to issues/PRs when they become unstale
+ labels-to-remove-when-unstale: stale,stale::closed
+
+ # Max number of operations per run
+ operations-per-run: ${{ secrets.STALE_OPERATIONS_PER_RUN || 100 }}
+ # Dry-run
+ debug-only: ${{ github.event.inputs.dryrun || false }}
+ # Order to get issues/PRs
+ ascending: true
+ # Delete branch after closing a stale PR
+ delete-branch: false
+
+ # Issues with these labels will never be considered stale
+ exempt-issue-labels: stale::recovered,epic
+ # Issues with these labels will never be considered stale
+ exempt-pr-labels: stale::recovered,epic
+ # Exempt all issues/PRs with milestones from stale
+ exempt-all-milestones: true
+ # Assignees on issues/PRs exempted from stale
+ exempt-assignees: mingwandroid
+
+ - name: Print outputs
+ run: echo ${{ join(steps.stale.outputs.*, ',') }}
=====================================
.pre-commit-config.yaml
=====================================
@@ -3,7 +3,7 @@ ci:
autofix_prs: false
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
- rev: v4.4.0
+ rev: v4.6.0
hooks:
- id: check-added-large-files
- id: check-ast
@@ -20,19 +20,19 @@ repos:
- id: check-yaml
exclude: conda.recipe/meta.yaml
- repo: https://github.com/asottile/pyupgrade
- rev: v3.3.1
+ rev: v3.15.2
hooks:
- id: pyupgrade
args: ["--py37-plus"]
- repo: https://github.com/PyCQA/isort
- rev: 5.12.0
+ rev: 5.13.2
hooks:
- id: isort
- repo: https://github.com/psf/black
- rev: 23.1.0
+ rev: 24.4.2
hooks:
- id: black
- repo: https://github.com/PyCQA/flake8
- rev: 6.0.0
+ rev: 7.0.0
hooks:
- id: flake8
=====================================
CHANGELOG.md
=====================================
@@ -1,5 +1,14 @@
[//]: # (current developments)
+## 0.10.0 (2024-06)
+
+* Use zip64 extensions when converting .tar.bz2 to .conda, if uncompressed size
+ is close to the 2GB ZIP64_LIMIT. (#79)
+
+## 0.9.0 (2023-07)
+
+* Respect umask when extracting files. [#65](https://github.com/conda/conda-package-streaming/pulls/65); [conda issue #12829](https://github.com/conda/conda/issues/12829).
+
## 0.8.0 (2023-05)
* Update transmute to use SpooledTemporaryFile instead of streaming directly to
=====================================
CODE_OF_CONDUCT.md
=====================================
@@ -1,6 +1,6 @@
# Conda Organization Code of Conduct
-> **Note**
+> [!NOTE]
> Below is the short version of our CoC, see the long version [here](https://github.com/conda-incubator/governance/blob/main/CODE_OF_CONDUCT.md).
# The Short Version
=====================================
HOW_WE_USE_GITHUB.md
=====================================
@@ -0,0 +1,334 @@
+<!-- absolute URLs -->
+[conda-org]: https://github.com/conda
+[sub-team]: https://github.com/conda-incubator/governance#sub-teams
+
+[project-planning]: https://github.com/orgs/conda/projects/2/views/11
+[project-sorting]: https://github.com/orgs/conda/projects/2/views/11
+[project-support]: https://github.com/orgs/conda/projects/2/views/12
+[project-backlog]: https://github.com/orgs/conda/projects/2/views/13
+[project-in-progress]: https://github.com/orgs/conda/projects/2/views/14
+
+[docs-toc]: https://github.blog/changelog/2021-04-13-table-of-contents-support-in-markdown-files/
+[docs-actions]: https://docs.github.com/en/actions
+[docs-saved-reply]: https://docs.github.com/en/get-started/writing-on-github/working-with-saved-replies/creating-a-saved-reply
+[docs-commit-signing]: https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits
+
+[infrastructure]: https://github.com/conda/infrastructure
+[workflow-sync]: https://github.com/conda/infrastructure/blob/main/.github/workflows/sync.yml
+[labels-global]: https://github.com/conda/infrastructure/blob/main/.github/global.yml
+
+<!-- relative URLs -->
+[workflow-cla]: /.github/workflows/cla.yml
+[workflow-issues]: /.github/workflows/issues.yml
+[workflow-labels]: /.github/workflows/labels.yml
+[workflow-lock]: /.github/workflows/lock.yml
+[workflow-project]: /.github/workflows/project.yml
+[workflow-stale]: /.github/workflows/stale.yml
+[labels-local]: /.github/labels.yml
+[labels-page]: ../../labels
+
+# How We Use GitHub
+
+This document seeks to outline how we as a community use GitHub Issues to track bugs and feature requests while still catering to development practices & project management (_e.g._, release cycles, feature planning, priority sorting, etc.).
+
+<!-- only include high-level topics or particularly noteworthy sections here -->
+**Topics:**
+
+ - [What is "Issue Sorting"?](#what-is-issue-sorting)
+ - [Issue Sorting Procedures](#issue-sorting-procedures)
+ - [Commit Signing](#commit-signing)
+ - [Types of Issues](#types-of-issues)
+ - [Standard Issue](#standard-issue)
+ - [Epics](#epics)
+ - [Spikes](#spikes)
+ - [Working on Issues](#working-on-issues)
+
+> [!NOTE]
+> This document is written in the style of an FAQ. For easier navigation, use [GitHub's table of contents feature][docs-toc].
+
+## What is "Issue Sorting"?
+
+> [!NOTE]
+> "Issue sorting" is similar to that of "triaging", but we've chosen to use different terminology because "triaging" is a word related to very weighty topics (_e.g._, injuries and war) and we would like to be sensitive to those connotations. Additionally, we are taking a more "fuzzy" approach to sorting (_e.g._, severities may not be assigned, etc.).
+
+"Issue Sorting" refers to the process of assessing the priority of incoming issues. Below is a high-level diagram of the flow of issues:
+
+```mermaid
+flowchart LR
+ subgraph flow_sorting [Issue Sorting]
+ board_sorting{{Sorting}}
+ board_support{{Support}}
+
+ board_sorting<-->board_support
+ end
+
+ subgraph flow_refinement [Refinement]
+ board_backlog{{Backlog}}
+
+ board_backlog-- refine -->board_backlog
+ end
+
+ subgraph flow_progress [In Progress]
+ board_progress{{In Progress}}
+ end
+
+ state_new(New Issues)
+ state_closed(Closed)
+
+ state_new-->board_sorting
+ board_sorting-- investigated -->board_backlog
+ board_sorting-- duplicates, off-topic -->state_closed
+ board_support-- resolved, unresponsive -->state_closed
+ board_backlog-- pending work -->board_progress
+ board_backlog-- resolved, irrelevant -->state_closed
+ board_progress-- resolved -->state_closed
+```
+
+### Why sort issues?
+
+At the most basic "bird's eye view" level, sorted issues will fall into the category of four main priority levels:
+
+- Do now
+- Do sometime
+- Provide user support
+- Never do (_i.e._, close)
+
+At its core, sorting enables new issues to be placed into these four categories, which helps to ensure that they will be processed at a velocity similar to or exceeding the rate at which new issues are coming in. One of the benefits of actively sorting issues is to avoid engineer burnout and to make necessary work sustainable; this is done by eliminating a never-ending backlog that has not been reviewed by any maintainers.
+
+There will always be broad-scope design and architecture implementations that the maintainers will be interested in pursuing; by actively organizing issues, the sorting engineers will be able to more easily track and tackle both specific and big-picture goals.
+
+### Who does the sorting?
+
+Sorting engineers are a conda governance [sub-team][sub-team]; they are a group of community members who are responsible for making decisions regarding closing issues and setting feature work priorities, among other sorting-related tasks.
+
+### How do items show up for sorting?
+
+New issues that are opened in any of the repositories in the [conda GitHub organization][conda-org] will show up in the "Sorting" tab of the [Planning project][project-planning]. There are two [GitHub Actions][docs-actions] workflows utilized for this purpose; [`.github/workflows/issues.yml`][workflow-issues] and [`.github/workflows/project.yml`][workflow-project].
+
+The GitHub Actions in the [`conda/infrastructure`][infrastructure] repository are viewed as canonical; the [`.github/workflows/sync.yml` workflow][workflow-sync] sends out any modifications to other `conda` repositories from there.
+
+### What is done about the issues in the "Sorting" tab?
+
+Issues in the ["Sorting" tab of the project board][project-sorting] are considered ready for the following procedures:
+
+- Mitigation via short-term workarounds and fixes
+- Redirection to the correct project
+- Determining if support can be provided for errors and questions
+- Closing out of any duplicate/off-topic issues
+
+The sorting engineers on rotation are not seeking to _resolve_ issues that arise. Instead, the goal is to understand the issue and to determine whether it is legitimate, and then to collect as much relevant information as possible so that the maintainers can make an informed decision about the appropriate resolution schedule.
+
+Issues will remain in the ["Sorting" tab][project-sorting] as long as the issue is in an investigatory phase (_e.g._, querying the user for more details, asking the user to attempt other workarounds, other debugging efforts, etc.) and are likely to remain in this state the longest, but should still be progressing over the course of 1-2 weeks.
+
+For more information on the sorting process, see [Issue Sorting Procedures](#issue-sorting-procedures).
+
+### When do items move out of the "Sorting" tab?
+
+Items move out of the ["Sorting" tab][project-sorting] once the investigatory phase described in [What is done about the issues in the "Sorting" tab?](#what-is-done-about-the-issues-in-the-sorting-tab) has concluded and the sorting engineer has enough information to make a decision about the appropriate resolution schedule for the issue. The additional tabs in the project board that the issues can be moved to include the following:
+
+- **"Support"** - Any issue in the ["Support" tab of the Planning board][project-support] is a request for support and is not a feature request or a bug report. Add the [`type::support`](https://github.com/conda/infrastructure/labels/type%3A%3Asupport) label to move an issue to this tab.
+- **"Backlog"** - The issue has revealed a bug or feature request. We have collected enough details to understand the problem/request and to reproduce it on our own. These issues have been moved into the [Backlog tab of the Planning board][project-backlog] at the end of the sorting rotation during Refinement. Add the [`backlog`](https://github.com/conda/infrastructure/labels/backlog) label to move an issue to this tab.
+- **"Closed"** - The issue was closed due to being a duplicate, being redirected to a different project, was a user error, a question that has been resolved, etc.
+
+### Where do work issues go after being sorted?
+
+Once issues are deemed ready to be worked on, they will be moved to the ["Backlog" tab of the Planning board][project-backlog]. Once actively in progress, the issues will be moved to the ["In Progress" tab of the Planning board][project-in-progress] and then closed out once the work is complete.
+
+### What is the purpose of having a "Backlog"?
+
+Issues are "backlogged" when they have been sorted but not yet earmarked for an upcoming release.
+
+### What automation procedures are currently in place?
+
+Global automation procedures synced out from the [`conda/infrastructure`][infrastructure] repo include:
+
+- [Marking of issues and pull requests as stale][workflow-stale], resulting in:
+ - issues marked as [`type::support`](https://github.com/conda/infrastructure/labels/type%3A%3Asupport) being labeled stale after 21 days of inactivity and being closed after 7 further days of inactivity (that is, closed after 30 inactive days total)
+ - all other inactive issues (not labeled as [`type::support`](https://github.com/conda/infrastructure/labels/type%3A%3Asupport) being labeled stale after 365 days of inactivity and being closed after 30 further days of inactivity (that is, closed after an approximate total of 1 year and 1 month of inactivity)
+ - all inactive pull requests being labeled stale after 365 days of inactivity and being closed after 30 further days of inactivity (that is, closed after an approximate total of 1 year and 1 month of inactivity)
+- [Locking of closed issues and pull requests with no further activity][workflow-lock] after 365 days
+- [Adding new issues and pull requests to the respective project boards][workflow-project]
+- [Indicating an issue is ready for the sorting engineer's attention][workflow-issues] by toggling [`pending::feedback`](https://github.com/conda/infrastructure/labels/pending%3A%3Afeedback) with [`pending::support`](https://github.com/conda/infrastructure/labels/pending%3A%3Asupport) after a contributor leaves a comment
+- [Verifying that contributors have signed the CLA][workflow-cla] before allowing pull requests to be merged; if the contributor hasn't signed the CLA previously, merging is be blocked until a manual review can be done
+- [Syncing out templates, labels, workflows, and documentation][workflow-sync] from [`conda/infrastructure`][infrastructure] to the other repositories
+
+## Issue Sorting Procedures
+
+### How are issues sorted?
+
+Issues in the ["Sorting" tab of the Planning board][project-sorting] are reviewed by issue sorting engineers, who take rotational sorting shifts. In the process of sorting issues, engineers label the issues and move them to the other tabs of the project board for further action.
+
+Issues that require input from multiple members of the sorting team will be brought up during refinement meetings in order to understand how those particular issues fit into the short- and long-term roadmap. These meetings enable the sorting engineers to get together to collectively prioritize issues, earmark feature requests for specific future releases (versus a more open-ended backlog), tag issues as ideal for first-time contributors, as well as whether or not to close/reject specific feature requests.
+
+### How does labeling work?
+
+Labeling is a very important means for sorting engineers to keep track of the current state of an issue with regards to the asynchronous nature of communicating with users. Utilizing the proper labels helps to identify the severity of the issue as well as to quickly understand the current state of a discussion.
+
+Each label has an associated description that clarifies how the label should be used. Hover on the label to see its description. Label colors are used to distinguish labels by category.
+
+Generally speaking, labels with the same category are considered mutually exclusive, but in some cases labels sharing the same category can occur concurrently, as they indicate qualifiers as opposed to types. For example, we may have the following types, [`type::bug`](https://github.com/conda/infrastructure/labels/type%3A%3Abug), [`type::feature`](https://github.com/conda/infrastructure/labels/type%3A%3Afeature), and [`type::documentation`](https://github.com/conda/infrastructure/labels/type%3A%3Adocumentation), where for any one issue there would be _at most_ **one** of these to be defined (_i.e._ an issue should not be a bug _and_ a feature request at the same time). Alternatively, with issues involving specific operating systems (_i.e._, [`os::linux`](https://github.com/conda/infrastructure/labels/os%3A%3Alinux), [`os::macos`](https://github.com/conda/infrastructure/labels/os%3A%3Amacos), and [`os::windows`](https://github.com/conda/infrastructure/labels/os%3A%3Awindows)), an issue could be labeled with one or more, depending on the system(s) the issue occurs on.
+
+Please note that there are also automation policies in place that are affected by labeling. For example, if an issue is labeled as [`type::support`](https://github.com/conda/infrastructure/labels/type%3A%3Asupport), that issue will be marked [`stale`](https://github.com/conda/infrastructure/labels/stale) after 21 days of inactivity and auto-closed after seven more days without activity (30 inactive days total), which is earlier than issues without this label. See [What automation procedures are currently in place?](#what-automation-procedures-are-currently-in-place) for more details.
+
+### What labels are required for each issue?
+
+At minimum, both `type` and `source` labels should be specified on each issue before moving it from the "Sorting" tab to the "Backlog" tab. All issues that are bugs should also be tagged with a `severity` label.
+
+The `type` labels are exclusive of each other: each sorted issue should have exactly one `type` label. These labels give high-level information on the issue's classification (_e.g._, bug, feature, tech debt, etc.)
+
+The `source` labels are exclusive of each other: each sorted issue should have exactly one `source` label. These labels give information on the sub-group to which the issue's author belongs (_e.g._, a partner, a frequent contributor, the wider community, etc.). Through these labels, maintainers gain insight into how well we're meeting the needs of various groups.
+
+The `severity` labels are exclusive of each other and, while required for the [`type::bug`](https://github.com/conda/infrastructure/labels/type%3A%bug) label, they can also be applied to other types to indicate demand or need. These labels help us to prioritize our work. Severity is not the only factor for work prioritization, but it is an important consideration.
+
+Please review the descriptions of the `type`, `source`, and `severity` labels on the [labels page][labels-page] prior to use.
+
+### How are new labels defined?
+
+Labels are defined using a scoped syntax with an optional high-level category (_e.g._, `source`, `tag`, `type`, etc.) and a specific topic, much like the following:
+
+- `[topic]`
+- `[category::topic]`
+- `[category::topic-phrase]`
+
+This syntax helps with issue sorting enforcement, as it helps to ensure that sorted issues are, at minimum, categorized by type and source.
+
+There are a number of labels that have been defined for the different repositories. In order to create a streamlined sorting process, label terminologies are standardized using similar (if not the same) labels.
+
+### How are new labels added?
+
+New **global** labels (_i.e._, labels that apply equally to all repositories within the conda GitHub organization) are added to [`conda/infrastructure`][infrastructure]'s [`.github/global.yml` file][labels-global]; new **local** labels (_i.e._, labels specific to particular repositories) are added to each repository's [`.github/labels.yml` file][labels-local]. All new labels should follow the labeling syntax described in ["How are new labels defined?"](#how-are-new-labels-defined). Global labels are combined with any local labels and these aggregated labels are used by the [`.github/workflows/labels.yml` workflow][workflow-labels] to synchronize the labels available for the repository.
+
+### Are there any templates to use as responses for commonly-seen issues?
+
+Some of the same types of issues appear regularly (_e.g._, issues that are duplicates of others, issues that should be filed in the Anaconda issue tracker, errors that are due to a user's specific setup/environment, etc.).
+
+Below are some boilerplate responses for the most commonly-seen issues to be sorted:
+
+<details>
+<summary><b>Duplicate Issue</b></summary>
+
+<!-- this is a code block, any links need to be embedded -->
+<pre>
+
+This is a duplicate of <b>[link to primary issue]</b>; please feel free to continue the discussion there.
+</pre>
+
+> [!WARNING]
+> Apply the https://github.com/conda/infrastructure/labels/duplicate label to the issue being closed and https://github.com/conda/infrastructure/labels/duplicate%3A%3Aprimary to the original issue.
+
+</details>
+
+<details>
+<summary><b>Requesting an Uninstall/Reinstall of <code>conda</code></b></summary>
+
+<!-- this is a code block, any links need to be embedded -->
+<pre>
+
+Please uninstall your current version of `conda` and reinstall the latest version.
+Feel free to use either the [miniconda](https://docs.anaconda.com/free/miniconda/)
+or [anaconda](https://www.anaconda.com/products/individual) installer,
+whichever is more appropriate for your needs.
+</pre>
+
+</details>
+
+<details>
+<summary><b>Redirect to Anaconda Issue Tracker</b></summary>
+
+<!-- this is a code block, any links need to be embedded -->
+<pre>
+
+Thank you for filing this issue! Unfortunately, this is off-topic for this repo.
+If you are still encountering this issue please reopen in the
+[Anaconda issue tracker](https://github.com/ContinuumIO/anaconda-issues/issues)
+where `conda` installer/package issues are addressed.
+</pre>
+
+> [!WARNING]
+> Apply the https://github.com/conda/infrastructure/labels/off-topic label to these issues before closing them out.
+
+</details>
+
+<details>
+<summary><b>Redirecting to Nucleus Forums</b></summary>
+
+<!-- this is a code block, any links need to be embedded -->
+<pre>
+
+Unfortunately, this issue is outside the scope of support we offer via GitHub;
+if you continue to experience the problems described here,
+please post details to the [Nucleus forums](https://community.anaconda.cloud/).
+</pre>
+
+> [!WARNING]
+> Apply the https://github.com/conda/infrastructure/labels/off-topic label to these issues before closing them out.
+
+</details>
+
+<details>
+<summary><b>Slow solving of conda environment</b></summary>
+
+<!-- this is a code block, any links need to be embedded -->
+<pre>
+Hi [@username],
+
+Thanks for voicing your concern about the performance of the classic dependency solver. To fix this, our official recommendation is using the new default "conda-libmamba-solver" instead of the classic solver (more information about the "conda-libmamba-solver" can be found here: https://conda.github.io/conda-libmamba-solver/getting-started/).
+
+In most cases "conda-libmamba-solver" should be significantly faster than the "classic" solver. We hope it provides you with a much better experience going forward.
+</pre>
+
+</details>
+
+
+In order to not have to manually type or copy/paste the above repeatedly, note that it's possible to add text for the most commonly-used responses via [GitHub's "Add Saved Reply" option][docs-saved-reply].
+
+## Commit Signing
+
+For all conda maintainers, we require commit signing and strongly recommend it for all others wishing to contribute to conda
+related projects. More information about how to set this up within GitHub can be found here:
+
+- [GitHub's signing commits docs][docs-commit-signing]
+
+## Types of Issues
+
+### Standard Issue
+
+TODO
+
+### Epics
+
+TODO
+
+### Spikes
+
+#### What is a spike?
+
+"Spike" is a term that is borrowed from extreme programming and agile development. They are used when the **outcome of an issue is unknown or even optional**. For example, when first coming across a problem that has not been solved before, a project may choose to either research the problem or create a prototype in order to better understand it.
+
+Additionally, spikes represent work that **may or may not actually be completed or implemented**. An example of this are prototypes created to explore possible solutions. Not all prototypes are implemented and the purpose of creating a prototype is often to explore the problem space more. For research-oriented tasks, the end result of this research may be that a feature request simply is not viable at the moment and would result in putting a stop to that work.
+
+Finally, spikes are usually **timeboxed**. However, given the open source/volunteer nature of our contributions, we do not enforce this for our contributors. When a timebox is set, this means that we are limiting how long we want someone to work on said spike. We do this to prevent contributors from falling into a rabbit hole they may never return from. Instead, we set a time limit to perform work on the spike and then have the assignee report back. If the tasks defined in the spike have not yet been completed, a decision is made on whether it makes sense to perform further work on the spike.
+
+#### When do I create a spike?
+
+A spike should be created when we do not have enough information to move forward with solving a problem. That simply means that, whenever we are dealing with unknowns or processes the project team has never encountered before, it may be useful for us to create a spike.
+
+In day-to-day work, this kind of situation may appear when new bug reports or feature requests come in that deal with problems or technologies that the project team is unfamiliar with. All issues that the project team has sufficient knowledge of should instead proceed as regular issues.
+
+#### When do I not create a spike?
+
+Below are some common scenarios where creating a spike is not appropriate:
+
+- Writing a technical specification for a feature we know how to implement
+- Design work that would go into drafting how an API is going to look and function
+- Any work that must be completed or is not optional
+
+## Working on Issues
+
+### How do I assign myself to an issue I am actively reviewing?
+
+If you do **not** have permissions, please indicate that you are working on an issue by leaving a comment. Someone who has permissions will assign you to the issue. If two weeks have passed without a pull request or an additional comment requesting information, you may be removed from the issue and the issue reassigned.
+
+If you are assigned to an issue but will not be able to continue work on it, please comment to indicate that you will no longer be working on it and press `unassign me` next to your username in the `Assignees` section of the issue page (top right).
+
+If you **do** have permissions, please assign yourself to the issue by pressing `assign myself` under the `Assignees` section of the issue page (top right).
=====================================
conda_package_streaming/__init__.py
=====================================
@@ -1 +1 @@
-__version__ = "0.8.0"
+__version__ = "0.10.0"
=====================================
conda_package_streaming/package_streaming.py
=====================================
@@ -5,12 +5,18 @@ Unpack conda packages without using a temporary file.
from __future__ import annotations
import bz2
+import os
import os.path
import tarfile
import zipfile
from enum import Enum
from typing import Generator
+# acquire umask taking advantage of import system lock, instead of possibly in
+# multiple threads at once.
+UMASK = os.umask(0)
+os.umask(UMASK)
+
try:
import zstandard
except ImportError:
@@ -30,6 +36,18 @@ class CondaComponent(Enum):
class TarfileNoSameOwner(tarfile.TarFile):
+ def __init__(self, *args, umask=UMASK, **kwargs):
+ """Open an (uncompressed) tar archive `name'. `mode' is either 'r' to
+ read from an existing archive, 'a' to append data to an existing
+ file or 'w' to create a new file overwriting an existing one. `mode'
+ defaults to 'r'.
+ If `fileobj' is given, it is used for reading or writing data. If it
+ can be determined, `mode' is overridden by `fileobj's mode.
+ `fileobj' is not closed, when TarFile is closed.
+ """
+ super().__init__(*args, **kwargs)
+ self.umask = umask
+
def chown(self, tarinfo, targetpath, numeric_owner):
"""
Override chown to be a no-op, since we don't want to preserve ownership
@@ -37,6 +55,16 @@ class TarfileNoSameOwner(tarfile.TarFile):
"""
return
+ def chmod(self, tarinfo, targetpath):
+ """
+ Set file permissions of targetpath according to tarinfo, respecting
+ umask.
+ """
+ try:
+ os.chmod(targetpath, tarinfo.mode & (0o777 - self.umask))
+ except OSError as e:
+ raise tarfile.ExtractError("could not change mode") from e
+
def tar_generator(
fileobj, tarfile_open=TarfileNoSameOwner.open, closefd=False
=====================================
conda_package_streaming/transmute.py
=====================================
@@ -1,14 +1,14 @@
"""
-Convert .tar.bz2 to .conda without temporary files.
+Convert .tar.bz2 to .conda
-Streams main `pkg-*` `.tar.zst` into open `ZipFile`, while buffering `info-*`
-`.tar.zst` in memory, writing it out at the end.
+Uses `tempfile.SpooledTemporaryFile` to buffer `pkg-*` `.tar` and `info-*`
+`.tar`, then compress directly into an open `ZipFile` at the end.
+`SpooledTemporaryFile` buffers the first 10MB of the package and its metadata in
+memory, but writes out to disk for larger packages.
-Works well for a typical ~10k `info-*`, but the conda format does not guarantee
-a small `info-*`.
-
-Conda packages created this way will also have `info-*` as the last element in
-the `ZipFile`, instead of the first for normal conda packages.
+Conda packages created this way have `info-*` as the last element in the
+`ZipFile`, instead of the first for `.conda` packages created with pre-2.0
+`conda-package-handling`.
"""
from __future__ import annotations
@@ -35,6 +35,10 @@ ZSTD_COMPRESS_THREADS = 1
CONDA_PACKAGE_FORMAT_VERSION = 2
+# Account for growth from "2 GB of /dev/urandom" to not exceed ZIP64_LIMIT after
+# compression
+CONDA_ZIP64_LIMIT = zipfile.ZIP64_LIMIT - (1 << 18) - 1
+
def transmute(
package,
@@ -103,16 +107,22 @@ def transmute(
conda_file.writestr("metadata.json", json.dumps(pkg_metadata))
with conda_file.open(
- f"pkg-{file_id}.tar.zst", "w"
+ f"pkg-{file_id}.tar.zst",
+ "w",
+ force_zip64=(pkg_size > CONDA_ZIP64_LIMIT),
) as pkg_file_zip, data_compress.stream_writer(
pkg_file_zip, size=pkg_size, closefd=False
) as pkg_stream:
shutil.copyfileobj(pkg_file._file, pkg_stream)
with conda_file.open(
- f"info-{file_id}.tar.zst", "w"
+ f"info-{file_id}.tar.zst",
+ "w",
+ force_zip64=(info_size > CONDA_ZIP64_LIMIT),
) as info_file_zip, data_compress.stream_writer(
- info_file_zip, size=info_size, closefd=False
+ info_file_zip,
+ size=info_size,
+ closefd=False,
) as info_stream:
shutil.copyfileobj(info_file._file, info_stream)
=====================================
tests/server.py
=====================================
@@ -1,6 +1,7 @@
"""
Test web server.
"""
+
import logging
import threading
import wsgiref.simple_server
=====================================
tests/test_extract.py
=====================================
@@ -1,4 +1,5 @@
import io
+import stat
import tarfile
from errno import ELOOP
@@ -33,13 +34,15 @@ def test_extract_all(conda_paths, tmp_path):
break
-def empty_tarfile(name):
+def empty_tarfile(name, mode=0o644):
"""
Return BytesIO containing a tarfile with one empty file named :name
"""
tar = io.BytesIO()
t = tarfile.TarFile(mode="w", fileobj=tar)
- t.addfile(tarfile.TarInfo(name=name), io.BytesIO())
+ tarinfo = tarfile.TarInfo(name=name)
+ tarinfo.mode = mode
+ t.addfile(tarinfo, io.BytesIO())
t.close()
tar.seek(0)
return tar
@@ -69,14 +72,25 @@ def test_oserror(tmp_path):
extract.extract_stream(stream(TarOSError), tmp_path)
+def stream(fileobj):
+ """
+ Like the tuples produced by part of conda-package-streaming.
+ """
+ yield (package_streaming.TarfileNoSameOwner(fileobj=fileobj), tarfile.TarInfo())
+
+
+def stream_stdlib(fileobj):
+ """
+ Like the tuples produced by part of conda-package-streaming.
+ """
+ yield (tarfile.TarFile(fileobj=fileobj), tarfile.TarInfo())
+
+
def test_slip(tmp_path):
"""
Fail if tarfile tries to put files outside its dest_dir (tmp_path)
"""
- def stream(fileobj):
- yield (tarfile.TarFile(fileobj=fileobj), tarfile.TarInfo())
-
tar = empty_tarfile(name="../slip")
with pytest.raises(exceptions.SafetyError):
@@ -99,3 +113,22 @@ def test_chown(conda_paths, tmp_path, mocker):
for tar, member in stream:
assert isinstance(tar, package_streaming.TarfileNoSameOwner), tar
break
+
+
+def test_umask(tmp_path, mocker):
+ """
+ Demonstrate that umask-respecting tar implementation works.
+
+ Mock umask in case it is different on your system.
+ """
+ mocker.patch("conda_package_streaming.package_streaming.UMASK", new=0o22)
+
+ tar3 = empty_tarfile(name="naughty_umask", mode=0o777)
+ extract.extract_stream(stream_stdlib(tar3), tmp_path)
+ mode = (tmp_path / "naughty_umask").stat().st_mode
+ assert mode & stat.S_IWGRP, "%o" % mode
+
+ tar3.seek(0)
+ extract.extract_stream(stream(tar3), tmp_path)
+ mode = (tmp_path / "naughty_umask").stat().st_mode
+ assert not mode & stat.S_IWGRP, "%o" % mode
=====================================
tests/test_streaming.py
=====================================
@@ -1,4 +1,6 @@
+import io
import json
+import tarfile
import pytest
@@ -31,3 +33,20 @@ def test_early_exit(conda_paths):
# stream_conda_info doesn't close a passed-in fileobj, but a
# filename should be closed.
assert found, f"index.json not found in {package}"
+
+
+def test_chmod_error(tmp_path, mocker):
+ """
+ Coverage for os.chmod() error handling.
+ """
+ with package_streaming.TarfileNoSameOwner(tmp_path / "test.tar", mode="w") as tar:
+ member = tarfile.TarInfo(name="file")
+ tar.addfile(member, io.BytesIO())
+
+ mocker.patch("os.chmod", side_effect=OSError)
+ with pytest.raises(tarfile.ExtractError):
+ # only logs a debug message if errorlevel<=1
+ with package_streaming.TarfileNoSameOwner(
+ tmp_path / "test.tar", errorlevel=2
+ ) as tar:
+ tar.extractall(tmp_path)
=====================================
tests/test_transmute.py
=====================================
@@ -88,12 +88,15 @@ def test_transmute_info_filter(tmpdir, testtar_bytes):
)
with open(Path(tmpdir, "test.conda"), "rb") as fileobj:
- for component, expected in (CondaComponent.info, {"info/expected"}), (
- CondaComponent.pkg,
- {
- "info/unexpected",
- "symlink",
- },
+ for component, expected in (
+ (CondaComponent.info, {"info/expected"}),
+ (
+ CondaComponent.pkg,
+ {
+ "info/unexpected",
+ "symlink",
+ },
+ ),
):
items = stream_conda_component("test.conda", fileobj, component)
assert {member.name for tar, member in items} == expected, items
@@ -129,3 +132,35 @@ def test_transmute_tarbz2_to_tarbz2(tmpdir, testtar_bytes):
out, testtar, strict=True
)
assert missing == mismatched == []
+
+
+def test_transmute_conditional_zip64(tmp_path, mocker):
+ """
+ Test that zip64 is used in transmute after a threshold.
+ """
+
+ LIMIT = 16384
+
+ for test_size, extra_expected in (LIMIT // 2, False), (LIMIT * 2, True):
+ mocker.patch("conda_package_streaming.transmute.CONDA_ZIP64_LIMIT", new=LIMIT)
+ mocker.patch("zipfile.ZIP64_LIMIT", new=LIMIT)
+
+ tmp_tar = tmp_path / f"{test_size}.tar.bz2"
+ with tarfile.open(tmp_tar, "w:bz2") as tar:
+ pkg = tarfile.TarInfo(name="packagedata")
+ data = io.BytesIO(os.urandom(test_size))
+ pkg.size = len(data.getbuffer())
+ tar.addfile(pkg, data)
+
+ info = tarfile.TarInfo(name="info/data")
+ data = io.BytesIO(os.urandom(test_size))
+ info.size = len(data.getbuffer())
+ tar.addfile(info, data)
+
+ out = transmute(str(tmp_tar), tmp_path)
+
+ with ZipFile(out) as e:
+ assert e.filelist[0].extra == b""
+ # when zip64 extension is used, extra contains zip64 headers
+ assert bool(e.filelist[1].extra) == extra_expected
+ assert bool(e.filelist[2].extra) == extra_expected
View it on GitLab: https://salsa.debian.org/med-team/conda-package-streaming/-/compare/4375d90f0ee36e4dbb09e5d1079b3669804751e9...1be1cd5c9d61571d0be6bb36b9508f2f8929b4dc
--
This project does not include diff previews in email notifications.
View it on GitLab: https://salsa.debian.org/med-team/conda-package-streaming/-/compare/4375d90f0ee36e4dbb09e5d1079b3669804751e9...1be1cd5c9d61571d0be6bb36b9508f2f8929b4dc
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/debian-med-commit/attachments/20240625/909d149d/attachment-0001.htm>
More information about the debian-med-commit
mailing list